Varnish: Scaling Django

November 1st, 2013 by ger

Last week, the new Quilmes site went live. We’ve been working hard on this project for several months and, knowing they will be promoting it using their Facebook fan page with near 4 millions likes, we knew we had to be prepared for a large number of users accessing the application in a short period of time.

There are many ways to improve and optimize a Django application (which we obviously implemented) but we needed something else for the potential volume we could received in this case.

Meet Varnish

Varnish is a caching HTTP reverse proxy that sits between your application and your users. In our case is actually between the load balancer and Django but the idea is the same. Varnish caches entire HTTP responses (pages, static files, even redirects) and, if the right conditions are met, the user can receive its response directly from it in a fraction of the time it would take Django to generate the same response.

How does Varnish know when to use the cached response and when to hit the application? Using Varnish Configuration Language, or VCL for short. The idea is that each application and domain is very different requiring specific rules for its configuration that can be expressed using this domain specific language.

Having said that, Varnish comes with a default configuration that caches any request GET or HEAD request with no cookies nor authentication header. A conservative but safe configuration that might be useful for some sites but not enough for even simple Django applications.

Our configuration

This is an example configuration for Django with a few tricks specific to this project to show how flexible VCL can be:

# Django Varnish configuration for unauthenticated views

# Django app is running in this same server on port 8008
backend default {
  .host = "localhost";
  .port = "8008";

sub vcl_recv {
  # unless sessionid is in the request, don't pass ANY cookies (referral_source, utm, etc)
  if (req.request == "GET" && (req.url ~ "^/static" || req.url ~ "^/media" || req.http.cookie !~ "sessionid")) {
    remove req.http.Cookie;

  # normalize accept-encoding to account for different browsers
  # see:
  if (req.http.Accept-Encoding) {
    if (req.http.Accept-Encoding ~ "gzip") {
      set req.http.Accept-Encoding = "gzip";
    } elsif (req.http.Accept-Encoding ~ "deflate") {
      set req.http.Accept-Encoding = "deflate";
    } else {
    # unkown algorithm
      remove req.http.Accept-Encoding;

sub vcl_fetch {
  # static/media files always cached
  if (req.url ~ "^/static" || req.url ~ "^/media") {
    unset beresp.http.set-cookie;
    return (deliver);

  # pass through for anything with a session/csrftoken set
  if (beresp.http.set-cookie ~ "sessionid" || beresp.http.set-cookie ~ "csrftoken") {
    return (hit_for_pass);
  } else {
    return (deliver);

What we are trying to do here is cache every unauthenticated request because those users are always seeing exactly the same page. Authenticated users are much more complex to handle because each user could potentially be looking at different versions of the same page.

What’s next

Ours is a relatively simple configuration that could be improved to implement a more aggressive caching strategy. We’ll probably do it in the future (and I’ll try to update this post when that happens) but for now this gives a big performance boost (our current hit ratio is 80%) without having to change the application at all. Further improvements would probably require changes to the application itself.

Varnish can do much more than what I showed you here such as acting as a load balancer with multiple backends and health checks, as a failover stragegy to keep returning cached content after their TTL if the backend is down (your users might not even know it), smart cache invalidation strategies.

More info

If you want to know more about Varnish and read about a few complex real-world examples you should checkout the following links:

Passbook Python library released

October 1st, 2012 by poli

Having worked with Python technologies for a while, at devartis we’ve been wanting to contribute some code to the community for some time. We just didn’t find the right opportunity until we started playing around with one of the new features bundled into iOS6: Passbook.

Passbook is an application that allows users to distribute, collect and easily retrieve movie tickets, boarding passes, loyalty cards, retail coupons, and other kind of passes on their iOS6 devices. The library we’ve written allows developers to create and read Passbook compatible pass files (.pkpass) in a very simple way. Here’s a code example:

from passbook.models import Pass, Barcode, StoreCard

cardInfo = StoreCard()
cardInfo.addPrimaryField('name', 'John Doe', 'Name')

passfile = Pass(cardInfo, \
    organizationName='<Your Organization Name>', \
    passTypeIdentifier='<>', \
    teamIdentifier='<Your Team ID>')

passfile.serialNumber = '1234567'
passfile.barcode = Barcode(message = 'Barcode message')
passfile.addFile('icon.png', open('images/icon.png'))
passfile.addFile('logo.png', open('images/logo.png'))

passfile.create( \
    'certificate.pem', \
    'key.pem', \
    'wwdr.pem', \
    '123456', \
    'test.pkpass') # Create and output the Passbook file (.pkpass)

Last Friday we submitted version 0.1.0 to Pypi which was downloaded by more than 150 developers just during the weekend! We have also started getting code contributions for new functionalities and some bugfixes through the project’s Github page, hence, version 0.2.0 was released today.

What’s next

We still need to define and develop a test suite that verifies that the .pkpass files are generated correctly: once that is done we will upgrade the status of the library to beta. Reading .pkpass files is also on the backlog. Be sure to follow Passbook at Github!

You can also check other open source contributions from devartis at

Technical bits

For more on how to generate the signed .pkpass file be sure to read the Building Your First Pass article on Passbook Programming Guide.

″Dale Aceptar 2012″ online!

June 18th, 2012 by ger

We are very proud to announce the release of our latest project: Dale Aceptar 2012. 1

Dale Aceptar 2012

‘Dale Aceptar’ is part of a program run by Sadosky Foundation aimed at fostering IT vocations. Using their words:

It is an ambitious program, its objective is to interest high school students all over Argentina about the opportunities the ICT technology offers. The idea is that many students get interested in taking up their studies at universities or junior colleges within this orientation. We must remember that, currently, the ICTs sector suffers a lack of human resources, as the yearly number of graduates is unable to meet the companies’ requests.

The project itself consists in an programming and animation contest using Alice (or its Spanish counterpart Rebecca), an innovative 3D programming environment that makes it easy to create an animation for telling a story, playing an interactive game, or a video to share on the web. The students will have the opportunity to learn, have fun and win prizes.

We implemented this site using our Python/Django stack which includes several preinstalled applications like Memcached 2 to improve responsiveness and performance, JS and CSS unification and minification 3 and fabric to automatize several tasks like deployments, backups and database migrations.

Additionally, a few Django apps where used for this project including django-cms to give the client an easy way to publish content to the site and pybb to implement forums and give the users a place to interact with each other.

So far, almost 14.000 high school students have registered in Dale Aceptar and are currently participating in the contest. If you know any student that might be interested in studying a ICT related career (perhaps he doesn’t know he might be interested), invite him to participate!

Get Adobe Flash player