This is the sad tale of hunting down … well, not a bug, but myself being an idiot.
Last post was about my new comic, Bubula². The website is a Django project and at the time of writing it is poorly optimized. One of the things to do is to optimize static files (e.g., minifying js and css). However, I have a poor deployment setup: Sync the source tree with the server and run collectstatic on the server. Instead I should collect static files locally, optimize the contents of the static folder and sync this folder. It is important, then, to have the same environment locally and on the server; otherwise, you may get static files from a wrong version of an app that will break your site. Therefore: use virtualenv!
So, I tried to do this. Just to check, I tried to compare the contents of the remote static folder and my local static folder. Something was wrong, even though I spent a lot of time ensuring that all packages was updated on my local and remote environment. It turned out that Django was collecting files from the apps installed in /usr/local/lib/python2.7/site-packages/ (installed before I decided to use virtualenv). But this folder was supposed not to be in PYTHONPATH!
I sat for hours trying various things, running “python2.7 manage.py collectstatic”, and still getting wrong results. Finally, because it was about 2 AM, I gave up and updated all global packages to get the correct results.
This morning I ran “which python2.7″ with the unexpected result:
But this was supposed to be <project path>/env/bin/python2.7! Guess what: this file did not exist; virtualenv only had a bin/python. I was fooled by a difference in the virtualenv on my server (same version – I’m really puzzled by this), which made a bin/python2.7 as well as a bin/python.
Morale: When hunting bugs the truth is often simple – the hard part is being smart enough to hunt in the right places.