self.machine

Today was a good day as i learned some valuable lessons about django and nginx.

1. ALWAYS close the database cursor in django, it can lead to some pretty wierd memory issues going forward.
2. FIND the most optimal number of database connections you initialize for you connection pooling. This will let you optimize on memory going forward.
3. ENSURE that you do not set a very high client_timeout, this means that if connections are not explicitly not closed by the client then the web server will not timeout. This results in bad memory behavior for the fcgi threads.

Well, this has solved my major issues with respect to kwippy. Also learned that GIFs are very fundamentally different than JPEGs which results in a lot of problem when used in the python imaging library. All in all a good day .

I know it would be simple for a lot of people reading this blog . But the easiest way to remote desktop into a windows machine using a X based linux distro is to install rdesktop.

http://www.rdesktop.org

For ubuntu and debian based distro, fire up the console and type

sudo apt-get install rdesktop

This would install the client.

rdesktop <server IP/domain>

This would fire up the windows remote desktop stuff.

Easiest way to get php with good looking urls running on your site . Works for drupal, wordpress and joomla.

server {
listen 80;
server_name www.domain.com;
index index.html index.htm index.php;
root /path/to/domain/files;
location / {
error_page 404 = //e/index.php?q=$uri;
}
location ~ .php$ {
include fastcgi_params;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME /path/to/domain/files$fastcgi_script_name;
}
access_log /usr/local/nginx/logs/domain.access_log;
error_log /usr/local/nginx/logs/domain.error_log;
}

This is a basic tutorial on how to get CodeIgniter facebook application on nginx (with the gotchas). The nginx configuration would be like the following

1
2
3
4
5
6
7
8
9
10
11
12
13

server {
        listen 80;
        server_name blah.com;
        location ~ /index.php/ {
                root           /home/production/blah;
                index  index.html index.htm index.php;
                include        conf/fcgi.conf;
                fastcgi_param  SCRIPT_FILENAME /home/production/fb_apps/quickdate/index.php;
                fastcgi_pass   127.0.0.1:9000;
        }
        access_log      /usr/local/nginx/logs/blah.access_log;
        error_log       /usr/local/nginx/logs/blah.error_log;
    }

The critical line is the fastcgi_params parameter, that changes the whole game. In the code Igniter application you need to add the following file in [app]/system/application/libraries/FB_controller.php

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

<?php
 
class FB_Controller extends Controller {
 
        // Facebook application key
        var $API_KEY = 'your api key';
 
        var $facebook;
        var $uid;
 
        /*
         * Custom Controller constructor.
         * Adds Facebook support.
         *
         */
        function FB_Controller() {
 
                parent::Controller();
 
                // Authentication key
                $secret = 'your secret key';
 
                // Prevent the 'Undefined index: facebook_config' notice from being thrown.
                $GLOBALS['facebook_config']['debug'] = NULL;
 
                // Create a Facebook client API object.
                $this->facebook = new Facebook($this->API_KEY, $secret);
                $this->uid = $this->facebook->require_login();
        }
}
?>

There are some significant changes in the configuration which is at [app]/system/application/config/config.php

1
2
3

$config['enable_query_strings'] = TRUE;
$config['subclass_prefix'] = 'FB_';
$config['uri_protocol'] = "REQUEST_URI";

Now in the application you should write controllers extending the above defined class , for example I modified the welcome controller.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

<?php
 
class Welcome extends FB_Controller {
        function Welcome() {
 
                parent::FB_Controller();
 
                // Check if the application has been added by the user
                try {
                        if (!$this->facebook->api_client->Users_isAppUser()) {
                                $this->facebook->redirect($this->facebook->get_add_url());
                                return;
                        }
                }
                catch (Exception $x) {
                        // Clear cookies for your application and redirect them to a login prompt
                        $this->facebook->expire_session();
                        $facebook->redirect($this->facebook->get_login_url());
                }
        }
        function index() {
        }
}

Fire this up and see it all running smoothly, this is basically compiled from all kinds of forum over the internet. Thought it will be useful for someone. In case of any issues contact me at me@dipankar.name.

In my experience with both rails and django, i would have to admit that a lot of things need to be improved at the core of these platforms so that developers can truly deploy a really fast production site. Let talk about what we did at kwippy to make it that much more faster than the default Django setup.

• Use memcached properly : The trick in getting speed is to cache all logged out pages and heavy caching of the user objects when logged in. For example we recently got our sessions into the memcached cloud to see a good speed jump.
• Database structuring : Django does a lot of joins and magic in it ORM, and has a built in caching layer. But the problem is that we developers structure our tables to our percieved needs … not necessarily for the way the ORM works. One way is to start writing custom SQL inside your views, another is to understand the ORM better. Your choice .
• Database connection pooling : It is pretty shocking to realize that Django does not do connection pooling, I have used DButils connection pooling for our needs. But IMHO it should be a default thing inside the application platform.
• SMTP is slow : Imagine the user filling up a form which is emailed to you, and your SMTP is down. There is a good chance that you will lose that data and the application will give a 500 . To alleviate that i have created a command queue where emails are not sent within the application and a daemon is doing all the dirty work. I will put it all out in the market , so that you guys can make you site that much more faster/robust.
• Pagination : To be honest, i still do not like django pagination. The problem was that it will getting all the objects and as the database is on a separate machine a lot of data was being transferred over the network. So i created my custom pagination which was faster than ObjectPaginator or Paginator.

Well there are all i could think of right now. Obviously there are some other optimizations that i keep doing …. and will keep writing about. Till the next time, feel free to contact me about these optimizations at me@dipankar.name.