Optimize Django for Performance

Optimize Django for Performance

Django is good stuff. I'm very convinced of that. However, there are always tradeoffs for awesomeness :) In Django, and in my opinion, this is performance. But a part of Django's awesomeness is its community, and boy did it take care of this problem the right way! You will most likely run into performance problems eventually when serving more complicated Django apps. Here are some personal guidelines I thought to be interesting to write up!

Cached template loaders

Add this to the settings.py file (or your production settings preferably):

TEMPLATES[0]['OPTIONS']['loaders'] = [('django.template.loaders.cached.Loader', TEMPLATES[0]['OPTIONS']['loaders'])]

This will modify the TEMPLATES setting to use a Cached Template loader, which will enable Django to load templates from the cache rather than from the filesystem. You can read more in the Django [The Django template language: for Python programmers |...](https://docs.djangoproject.com/en/1.10/ref/templates/api/#django.template.loaders.cached.Loader 'The Django template language: for Python programmers | Django documentation | Djangoundefined)

Enable Django's caching framework

Django is pretty good at caching. You can read more about it here: [Django’s cache framework | Django documentation | Django](https://docs.djangoproject.com/en/1.10/topics/cache/ 'Django’s cache framework | Django documentation | Djangoundefined). At Luminum Solutions, we use Redis for all our caching. It's fast, actively developed, well supported, easy to configure, scalable, and very manageable in production.

Django Redis

The first best quickwin is [django-redis.](https://niwinz.github.io/django-redis/latest/ 'django-redis documentationundefined) You can hook it into Django's native caching framework, so it's **very **easy to set up. What's also pretty cool is that Django Redis does this thing called connection pooling in the background. I've never seen too many connections cause (performance) problems with a cache, but connection pooling is generally a good thing. Especially in database-land where PgBouncer can come to the quick rescue, but that's another story :-)

Django Cacheops

The second Redis based cache is [Suor/django-cacheops: A slick ORM cache.](https://github.com/Suor/django-cacheops 'GitHub - Suor/django-cacheops: A slick ORM cache with automatic granular event-driven invalidation.undefined)

You can use it in combination with Django Redis, as Django Cacheops serves a different purpose: it's an ORM cache. This means that you can hook it into Django's ORM to cache queries. There is some overlap in functionality. I personally prefer to use Django-Redis for most tasks like view caching or template fragment caching ([Django’s cache framework | Django documentation | Django](https://docs.djangoproject.com/en/1.10/topics/cache/ 'Django’s cache framework | Django documentation | Djangoundefined)), as it's more tightly integrated with Django itself.

Naturally, there are tons of other cache storages like Memcached, the database, filesystem, and local-in-memory. If you're running a simple website that still requires good performance, the local in-memory options are a very good option.

General notes on caching (the gotchas)

Most of the gotchas around caching have something to do with invalidation. It can be quite a user experience killer if they end up with an untranslated menu, or with someone else's profile even.

There's one simple good habit that will enable you to set up good caching configurations; always ask yourself what can change in the entity you are considering caching. In the case of template or query set caching, which is where you'll most likely end up, it's good to list (maybe even write down) all the variables that should imply a change in content.

Security considerations

As most security priorities involve sensitive data, auth is the first thing to look at when keeping security and privacy in mind.

  • Never cache things like passwords. Preferably, just steer clear of exerting fine-grained cache control over the auth system. I personally try to focus on getting the database to only worry about auth, as all other things can usually be cached more easily and without the potential of a headache :-)
  • Don't cache auth-related templates like profile pages. When you can prevent caching profile pages (or even fragments), try to do so. This also follows the database-for-auth philosophy!