Heroku, Django, and Static Files

I've been continuing to mess with Heroku and it's been a blast to just have someplace to host on that's so bloody easy to get going. As part of this I've also decided to give Django a second chance.

My History with Django

A few years back I used Django a lot when working on Curse.com and CurseForge. We were using Django 0.7, and in fact due to some poor planning by certain members of the dev team we worked off of a random subversion revision. It was then forked and patched with a lot of custom changes that we depended heavily on and all without any record keeping of what version we were currently based on. It became such a monumental task to try to upgrade that we eventually stopped trying.

That paralysis ended up costing a lot in the long run. We were unable to upgrade and as a result were no longer able to take advantage of the growth and maturation of the platform. Eventually we relaunched Curse.com as a C# .Net website and CurseForge eventually became based on a lower level stack based on Werkzeug, Jinja2, SQLAlchemy, and WTForms.

Curse.com is a huge site, it does millions of page views on an average day, and on a busy day (World of Warcraft patch days for example) it can do tens of millions. At the time we ran into a lot of issues with scaling a Django site, but as I said that was on a random development version in the 0.7 series. The whole experience left me with a pretty bad taste in my mouth as it where. Recently I've heard it's gotten a lot better and I wanted to see for myself.

And today

Now Django is on 1.4 and a lot of things have changes, both small and subtle and major and obvious. One of the more significant changes I've seen so far is how static files are handled. They've added and abstracted storage backend system that's actually pretty nice.

The default backend works with the filesystem. It takes discovered files and puts them into a folder determined by the STATIC_ROOT setting. This is kinda messy on Heroku, but can absolutely be used. Recently Heroku even started running manage.py collectstatic for you. Despite that being a viable solution, I don't particularly care fore that.

I found a lot of people talking about using S3 and django-storages for static files. So I gave it a go and was able to get all my files collected directly into S3 pretty quickly, but there was some things I didn't like. I found, for example that the admin interface didn't play well with using S3's vanity domains, and the documentation on django-storages was ungodly frustrating. It only documented a handful of config options, but after inspecting the code it actually has twenty.

I wanted something simpler, and so I did the not-so-civically-responsible thing, and wrote my own simple version of the S3Storage. So after a few evenings I have a very simple, working backend. It does make a lot more assumptions, and it just plain out doesn't bother having a lot of settings. I've put it up on pypi, and I'm going to be putting some documentation here soon.