In this post, you will learn how to run tests in django with an existing database.

Legacy code use case

Sometimes we have to work with some legacy systems and it can be painful. Breaking things is usually unacceptable, but errors happen and we should catch them as early as possible. The best way to do it is having a good test coverage. It's especially important in dynamic, interpreted languages such as python or ruby, because it's easier to miss error without a strict compiler. So, tests are awesome and we should use them.

However, we can be in a situation where we don't even have a working continuous integration testing and our manage.py test throws an error during creation of test database.

Using an alternatively created database

If we can't build a test database using standard django methods, testing get a bit more complicated. Usually we can recreate a fresh database somehow, using raw sql by copying database layout from the production database, inserting some initial data. It's messy and ugly, but it works.

Let's assume that we already have an automated script for building a database. We can make also a database for our tests and delete it afterwards.

Overriding test runner to use provided database in tests

Now there is a question: how to make django test runner work with it?

You can read in django advanced testing docs how to do it, that there is a setting for that, but actually it's something different. Mirror test database in django is a setting useful for master-slave setups.

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'myproject',
        'HOST': 'dbprimary',
         # ... plus some other settings
    },
    'replica': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'myproject',
        'HOST': 'dbreplica',
        'TEST_MIRROR': 'default'
        # ... plus some other settings
    }
}

We would like to use test_mirror database for tests. The closest solution I could find on stack overflow was to create an alternative test runner. However the best and accepted solution, this answer comes from 2011... and is simply a bit outdated.

In my particular use case I worked with the old django 1.5, in which you should look at DjangoTestSuiteRunner class and in newer django it's DiscoverRunner class.

Here is my solution (for django 1.5, for newest django inherit after DiscoverRunner).

mirror_test_db_test_runner.py

from django.test.simple import DjangoTestSuiteRunner
from django.conf import settings


class MirrorTestDBTestSuiteRunner(DjangoTestSuiteRunner):

    def setup_databases(self, **kwargs):
        from django.db import connections
        old_names = []
        mirrors = []
        for alias in connections:
            connection = connections[alias]
            # If the database has a test mirror
            # use it instead of creating a test database.
            mirror_alias = connection.settings_dict['TEST_MIRROR']

            if mirror_alias:
                mirrors.append((alias,
                                connections[alias].settings_dict['NAME']))
                connections[alias].settings_dict['NAME'] = (
                        connections[mirror_alias].settings_dict['NAME'])
            elif not connection.settings_dict.get('BYPASS_CREATION', False):
                old_names.append((connection,
                                  connection.settings_dict['NAME']))
                connection.creation.create_test_db(self.verbosity)

        return old_names, mirrors

    def run_tests(self, test_labels, extra_tests=None, **kwargs):
        # This test runner tried to run test on everything,
        # even on the django itself, this is a hacky workaround ;)
        super(MirrorTestDBTestSuiteRunner, self).run_tests(
            settings.TESTED_APPS, extra_tests, **kwargs)

Basically there are three options.

  1. Use TEST_MIRROR for database which has it specified.
  2. Omit creation of TEST_MIRROR, because it's provided and django shouldn't try to recreate it.
  3. Treat databases without BYPASS_CREATION and TEST_MIRROR in the usual way.

The test runner assumes that we made appropriate changes in our django settings.

It can look like this:

settings.py:

# Use custom test runner
TEST_RUNNER = 'mirror_db_test_runner.MirrorTestDBTestSuiteRunner'

...

# Whitelist what exactly do we want to test
TESTED_APPS = (
    'my_app',
    'another_app',
)

# `TEST_MIRROR` and `BYPASS_CREATION` settings for custom test runner

DATABASES = {
    u'default': {
        u'ENGINE': u'your_engine',
        u'NAME': u'name',
        ...
        u'TEST_MIRROR': 'test_mirror'
    },
    # setting of database used for testing (should mirror the `default` one 
    u'test_mirror': {
        u'ENGINE': u'django.db.backends.postgresql_psycopg2',
        u'NAME': u'test_mirror',
        ... # same settings as default
        u'BYPASS_CREATION': True,
    },
}

If you have an existing test_mirror database, django should use it in tests. You're now one step closer to sane codebase.