7 years of Django in 7-ish days

Spring was quite an "interesting time" for my personal project: WaterOnMars.

Indeed I started to work on adding a new feature (a first in a while but maybe the topic of another post) but each time I was pushing or deploying code I was suddenly getting back warnings unrelated to my changes but pointing at core components like, err... Python or Django versions being deprecated.

So kudos for Python and github developers for making a clever use of warnings and, yes, I admit that using Python2.7 (ending its life in 2020) and Django1.4 (published 7 years ago) in 2019 is lame.

So... migrations !

All migrations need a path

The main target was to upgrade WaterOnMars to a newer version of Django (at the time this was 2.2) and using Python3.

Funny enough Django2.2 only works with Python3 and Django1.4 doesn't work at all on Python3.

Luckily the Django project drew most of the path for me as the 1.x versions started to support Python3 (with v.1.5) and kept support for Python2 until the end (v. 1.11).

So the main migration path was: 1. migrate from Django1.4 to Django1.11 2. migrate from Python2 to Python3 3. migrate from Django1.11 to Django2.2

Now I must say this was a surprisingly smooth sailing BUT also one were the obstacles were absolutely not where I was expecting them.

Caveats (of course)

For starters the fact that Django makes breaking changes between minor versions changes radically the naive expectations one may have when looking at this simple path.

South migation

(punned forced on me)

The first real caveat though was the deprecation of South the database migration library that was somehow integrated as a "standard" Django module but with significant incompatible changes.

Even if the probject is now frozen, it seemed safer to me though, to first upgrade South to its very latest version, before jumping in the official South to pure django migration.

Minor breaking changes toward 1.11

For the rest of the minor versions breaking changes, they were in a sufficiently small number that I could deal with them in a single changeset.

You may think that it's not very good practice to put everything in a big commit and I usually agree, but being alone so far on the project, I had no interest in testing the full app for each minor version of Django and prefered doing the test and fixes only once with Django1.11.

Most changes were straightforward and with no drawbacks I could foresee.

The only tedious one was the change of contract for finding urls by "reversing" the view function as it happens I was massively using a now obsolete way to do so. But I believe the new code is cleaner.

Eventually there were only two changes for which I'm still not sure how much of a gain it was to do so (beyond just being compatible of course).

First is the change of contract for the (input) request object that contains a POST dictionary only if it has been generated from a form. Meaning that a simpler POST request's content would not show up anymore in this member variable but only in the request body !

Second is the change of the DB migration library: 7years ago I was very happy to find that a library was there to handle DB migrations for me. But when, this spring, I realized I had to migrate the migration library, I felt a little dumb and started to ask myself questions I initially wanted to avoid about how all this is working internally.

Migration to Python3

This was mostly uneventful thanks to the 2to3 utility.

The only surprise was the incompatibility of fabric1 the tool I'm using to define the various lifecycle tasks (test, deploy, update schema ...) of the project. And since fabric2 was a major re-design and split of whatever made fabric1 what it was, upgrading the script was actually the most tedious part of the Python migration.

The rest was a breeze as can be checked in the commit message of this changeset.

A minor major migration

Migrating from Djang1.11 to 2.2 ended up much easier than migrating between the 1.x minor versions. This is of course due to the fact that it's really only 3 minor versions steps of Django as there was no "bigger jump" between the 1.11 qnd 2.0 than the ones you get between some other minor versions.

Here's the changeset with few details.

Kudos anyway

Django's excellent documentation: I'm not hellbent against minor version breaking changes even if it's a bit counter-intuitive, but these were very well documented and easy to apply.

Python's 2to3 tool and Python3 in general: I'm late in the game I know but still I appreciate that. In particular Python3's string handling has a huge postive impact on a web application such as this one.

Warnings from Github ("push warnings" on deprecated libraries) and Python (2.7 EOL warning): they ended-up being a great motivator, and proved much more efficient in that sense than what I would have guessed if anybody had had the bad idea to ask for my opinion ;)