self.machine

When you are using a platform like django you realise how slow sessions can get when you are using the database as a backend. The problem of using a memory cache like memcached is the fact that when you restart the server to refresh the cache or remove stale objects, the problem is that you lose your sessions data and a lot of people using your site get logged out. The only solution to this problem is to use 2 memcached instances , one for your regular python objects and another for your sessions objects … this is not a default feature in Django. So here is the solution to this particular problem.

In your project directory create the following files, the first file is basically a copy of a core django file (contrib/sessions/backends/cache.py) and the second is a file where the class gets initialized (its not necessary , but a good example).

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55

from django.contrib.sessions.backends.base import SessionBase, CreateError
from kwippyproject.session_cache import cache
 
class SessionStore(SessionBase):
    """
    A cache-based session store.
    """
    def __init__(self, session_key=None):
        self._cache = cache
        super(SessionStore, self).__init__(session_key)
 
    def load(self):
        session_data = self._cache.get(self.session_key)
        if session_data is not None:
            return session_data
        self.create()
        return {}
 
    def create(self):
        # Because a cache can fail silently (e.g. memcache), we don't know if
        # we are failing to create a new session because of a key collision or
        # because the cache is missing. So we try for a (large) number of times
        # and then raise an exception. That's the risk you shoulder if using
        # cache backing.
        for i in xrange(10000):
            self.session_key = self._get_new_session_key()
            try:
                self.save(must_create=True)
            except CreateError:
                continue
            self.modified = True
            return
        raise RuntimeError("Unable to create a new session key.")
 
    def save(self, must_create=False):
        if must_create:
            func = self._cache.add
        else:
            func = self._cache.set
        result = func(self.session_key, self._get_session(no_load=must_create),
                self.get_expiry_age())
        if must_create and not result:
            raise CreateError
 
    def exists(self, session_key):
        if self._cache.get(session_key):
            return True
        return False
 
    def delete(self, session_key=None):
        if session_key is None:
            if self._session_key is None:
                return
            session_key = self._session_key
        self._cache.delete(session_key)

1
2
3
4
5
6

from django.core.cache.backends.memcached import CacheClass
from django.conf import settings
 
scheme, rest = settings.SESSION_CACHE.split(':', 1)
host = rest[2:-1]
cache = CacheClass(host,{})

1
2

SESSION_ENGINE = "kwippyproject.session_backend"
SESSION_CACHE = 'memcached://127.0.0.1:11200/'

This is a small tutorial on how to use HTTP authentication for your site. Firstly you need to copy this file to httpauth.py and place it at /httpauth.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111

 
import base64
from django.contrib.auth.models import User
from django.http import HttpResponse
from django.contrib.auth import authenticate, login
 
#############################################################################
#
def view_or_basicauth(view, request, test_func, realm = "", *args, **kwargs):
    """
    This is a helper function used by both 'logged_in_or_basicauth' and
    'has_perm_or_basicauth' that does the nitty of determining if they
    are already logged in or if they have provided proper http-authorization
    and returning the view if all goes well, otherwise responding with a 401.
    """
    if test_func(request.user):
        # Already logged in, just return the view.
        #
        return view(request, *args, **kwargs)
 
    # They are not logged in. See if they provided login credentials
    #
    if 'HTTP_AUTHORIZATION' in request.META:
        auth = request.META['HTTP_AUTHORIZATION'].split()
        if len(auth) == 2:
            # NOTE: We are only support basic authentication for now.
            #
            if auth[0].lower() == "basic":
                uname, passwd = base64.b64decode(auth[1]).split(':')
                email_user = User.objects.filter(email=uname)
                if not email_user:
                    username = uname
                else:
                    username = email_user[0].username
                user = authenticate(username=username, password=passwd)
                if user is not None:
                    if user.is_active:
                        login(request, user)
                        request.user = user
                        return view(request, *args, **kwargs)
 
    # Either they did not provide an authorization header or
    # something in the authorization attempt failed. Send a 401
    # back to them to ask them to authenticate.
    #
    response = HttpResponse()
    response.status_code = 401
    response['WWW-Authenticate'] = 'Basic realm="%s"' % realm
    return response
 
#############################################################################
#
def logged_in_or_basicauth(realm = ""):
    """
    A simple decorator that requires a user to be logged in. If they are not
    logged in the request is examined for a 'authorization' header.
 
    If the header is present it is tested for basic authentication and
    the user is logged in with the provided credentials.
 
    If the header is not present a http 401 is sent back to the
    requestor to provide credentials.
 
    The purpose of this is that in several django projects I have needed
    several specific views that need to support basic authentication, yet the
    web site as a whole used django's provided authentication.
 
    The uses for this are for urls that are access programmatically such as
    by rss feed readers, yet the view requires a user to be logged in. Many rss
    readers support supplying the authentication credentials via http basic
    auth (and they do NOT support a redirect to a form where they post a
    username/password.)
 
    Use is simple:
 
    @logged_in_or_basicauth
    def your_view:
        ...
 
    You can provide the name of the realm to ask for authentication within.
    """
    def view_decorator(func):
        def wrapper(request, *args, **kwargs):
            return view_or_basicauth(func, request,
                                     lambda u: u.is_authenticated(),
                                     realm, *args, **kwargs)
        return wrapper
    return view_decorator
 
#############################################################################
#
def has_perm_or_basicauth(perm, realm = ""):
    """
    This is similar to the above decorator 'logged_in_or_basicauth'
    except that it requires the logged in user to have a specific
    permission.
 
    Use:
 
    @logged_in_or_basicauth('asforums.view_forumcollection')
    def your_view:
        ...
 
    """
    def view_decorator(func):
        def wrapper(request, *args, **kwargs):
            return view_or_basicauth(func, request,
                                     lambda u: u.has_perm(perm),
                                     realm, *args, **kwargs)
        return wrapper
    return view_decorator

You can use it in the view like this

1
2
3
4
5

 
from httpauth import *
@logged_in_or_basicauth()
def fu(request,type):
     pass

So this is dead simple and allows you to use simple HTTP authentication for your views. For any more help on this catch me at me@dipankar.name.

I was thinking about the book *The long tail* , its a great read even if it is  bit old. The ideas are very much relevant in today’s businesses and will continue to be as long as the Gaussian bells are around. After some discussions with my friends i chanced upon this interesting idea that using the *long tail model* you can effectively change the consumer usage patterns, much like the Heisenberg principle in physics. Lets take a simple example to show this :

Not too many people want to watch Kurosawa’s Rashomon, so imagine X (a company which rents out movies) keeps this following the *long tail* concept. Some how it gets picked up by some movie person , who recommends it …. this will create a network growth effect which can potentially skew the graphs on which the theory is based (conclusion : rashomon may become a *super* rental movie for X).

A movie which was relatively lesser know gets watched more because its niche and with time it skews the initial movie rental graphs in its favor.  So the curve changes because the movie gets stocked, so the ability to predict the number of copies for a certain movie becomes way more difficult with time.

Relevant links:

http://en.wikipedia.org/wiki/Observer_effect

http://en.wikipedia.org/wiki/The_Long_Tail

Hmmm, unlike what most people have to say I have started to feel that Indians working in “outsourced” development teams are getting a real bad deal. In my recent experience, i have felt that in spite of all the big talk on how India has become costlier blah blah …. its my firm belief that there is very good quality of work being done here ….

So quality development work on cutting edge platforms is still termed as outsourcing, that was a rude shock to a guy like me . I have seen some great stuff coming out, the only point is that few people realize that the work is being done by Indians ….. maybe when that changes we will be a better appreciated lot …. I was talking to some person who made me understand that nobody had any idea about what the team had contributed to slideshare.net, and how my bosses have done a great job …. I was really perplexed, but then the realization dawned upon me that conceptualization / abstract talk is way higher up the value chain (i think thats all bull crap … but then again i am an engineer he he) …..

To be honest, some of the implementation details were awesomely creative …. and a lot of thought went behind them. Slideshare has been built in India, not implemented …. even if we all don’t know of the *names* behind them that is something the world should know about.

So this post is devoted to my conception of a startup and the differences (or at least perceived ones) it has from a big firm. (Warning: These are very subjective differences)

1. Startups are built *for* people who are ambitious and driven , thus this gives them the momentum going forward even if resources are scarce.
2. From the above point it can be agreed easily that people who work at such places are to some extent *different*. Some will be motivated by money , some by thirst for knowledge etc etc.
3. The critical aspect of all this is that like artists at work, most developers have their zen moments (or aha moment as vikas would say), i believe they need care and nurturing from their patrons (aka managers).
4. Startups start with the vision of changing the world, and that feeling is necessary for exponential(to some extent non-human … non-linear ) growth pattern.

These differences are what make a startup a temple to learn and grow in as compared to big corporations where you cannot have the same effect as an individual. I think that now-a-days the term start-up is thrown around rather randomly … maybe there is a need for some certification on that end.