Recently I was working on new exciting part of syncano-platform and I wanted to create a piece of api that would allow for filtering resources with many to many relationships.

I wanted to have a resource that can be tagged.

Adding such tags should be RESTful and straightforward. Also I wanted to add the ability to search using tags. My go to Rest library is Django Rest Framework.

Currently the most popular solution for tagging in django is django-taggit, however it doesn't integrate with Django Rest Framework well and heavily uses ContentTypes. In my current project at work we don't use ContentTypes at all.

It's also a pretty old library and has support for old django, south migrations, stuff that we just don't need in new projects.

I've read in a blog post about django-taggit:

Tags can be hard if you try to implement them yourself, so don't try and just use this library from the start.

And well, I disagree with this quote, how hard can it can be?

I sometimes prefer to write a simple code, well tailored to my use case, that install another library with all its dependencies. If library is poorly written you'll waste a lot of time debugging its strange behaviors and fighting with it when trying to tailor it your custom requirements. (I'm looking at you python social auth)

I decided to just do it myself. We will now distill this experience by building a simple app with two resources and REST API for them. Here is a code for a whole app.

App Ingredients

  • python 3.4
  • django 1.8
  • Django Rest Framework 3.1
  • django-filter
  • django-cors-headers 1.1.0

For simplicity I will use sqlite as a DB.

Data models

Let's imagine that we're building a very simplified version of PYPI. We're indexing libraries, we store their names, websites, descriptions and tags.

User can then search libraries using names, descriptions and tags, they can even add new tags to them.

How we will build it? We will have two resources

  • tags
  • libraries

In django, models are defined in models.py

models.py

from django.db import models


class Tag(models.Model):
    """Tag for data. Every tag has unique text.
    """
    text = models.CharField(max_length=64, unique=True)

    def __str__(self):
        return 'Tag[id: {id}, text: {text}]'.format(
            id=self.id, text=self.text)


class Library(models.Model):
    """Library represents a piece of software. It has its website url,
    name, description and as many tags as you like.
    """

    # django has a nice field that validates URLs
    url = models.URLField()

    # name should be in a slug form
    name = models.SlugField(max_length=80)
    description = models.CharField(max_length=256, blank=True)
    tags = models.ManyToManyField(Tag, related_name='libraries')

    def __str__(self):
        return 'Library[id: {id}, name: {name}]'.format(
            id=self.id, name=self.name)

And we will create nice RESTy api for them. It will be possible to add tags to libraries and vice versa.

To use django rest framework with those models, we need at three things:

  • serializers
  • views
  • routing

I usually start with serializers, serializer will translate our models to serialized date (it can be other way round too). I'm going to use simple serializers based on models directly. DRF has ModelSerializers for that.

serializers.py

from rest_framework import serializers

from .models import Library, Tag


class TagSerializer(serializers.ModelSerializer):
    libraries = serializers.SlugRelatedField(
        many=True,
        read_only=True,
        slug_field='name'
    )

    class Meta:
        model = Tag


class LibrarySerializer(serializers.ModelSerializer):
    tags = serializers.SlugRelatedField(
        many=True,
        queryset=Tag.objects.all(),
        slug_field='text'
    )

    class Meta:
        model = Library

The only tricky part there was to define SlugRelatedFields correctly. You can read more about related fields in DRF here.

With serializers in place, we can move to views part. Views let our app to communicate with the rest of the world.

In DRF you can either use views or viewsets. Views are simpler to understand, because they have the same methods as http verbs. Viewsets simplify lots of things and they are on higher abstraction level, so instead of post, you have create method.

Also viewsets combine very well with routers and usually using viewset results in using much less code.

Here I will use ModelViewSets, that gives us full CRUD functionality for free.

views.py

from rest_framework import viewsets

from .models import Library, Tag
from .serializers import LibrarySerializer, TagSerializer


class TagViewSet(viewsets.ModelViewSet):
    queryset = Tag.objects.all()
    serializer_class = TagSerializer


class LibraryViewSet(viewsets.ModelViewSet):
    queryset = Library.objects.all()
    serializer_class = LibrarySerializer

Simple, isn't it?

Now, let's connect views to urls.

urls.py

from rest_framework.routers import DefaultRouter

from .views import LibraryViewSet, TagViewSet

router = DefaultRouter()
router.register(r'libraries', LibraryViewSet)
router.register(r'tags',TagViewSet)
urlpatterns = router.urls

That's actually all we need to create DRF browsable API for those resources.

Imgur

Adding filtering

We can add new things now, but finding something is currently a pain.

Let's implement filtering! We will enable filtering of libraries by:

  • tags
  • name
  • description

DRF has a pretty good support for filtering. You can read more about it here.

Even though support is good, we still have to do some things:

  • create a custom filter
  • enable filtering Backend

Let start from creating a custom filter. We will put in in separate file, so it will be easier to find.

filters.py

import django_filters

from .models import Library, Tag


class LibraryFilter(django_filters.FilterSet):
    # default for CharFilter is to have exact lookup_type
    name = django_filters.CharFilter(lookup_type='icontains')
    description = django_filters.CharFilter(lookup_type='icontains')

    # tricky part - how to filter by related field?
    # but not by its foreign key (default)
    # `to_field_name` is crucial here
    # `conjoined=True` makes that, the more tags, the more narrow the search
    tags = django_filters.ModelMultipleChoiceFilter(
        queryset=Tag.objects.all(),
        to_field_name='text',
        conjoined=True,
    )

    class Meta:
        model = Library
        fields = ['name', 'description', 'tags']

To create a custom filter, we had to inherit after django_filters.FilterSet, define our model in Meta and define how each of filters should behave.

django_filters offers few types of lookup_types for example: exact - that would match an exact string, 'contains', that would returns matches that contain given string, you can add i before those lookup_types, to make them ignore case.

The only tricky part is filtering by related field using its field, different than foreign key. I've seen some examples with filtering by related fields using foreign key, but I couldn't find in any docs, how to do it with different field (in our case - tag text).

Eventually I had to dig into source of django fields to find it out. It's enough to set to_field_name='field_name' and it should work.

To connect filter with view, you have to add following too lines to the view:

filter_backends = (filters.DjangoFilterBackend,)
filter_class = LibraryFilter

You can also set filtering backend in django app settings if you like.

Here is how it should look like in context:

views.py

from rest_framework import filters, viewsets

from .filters import LibraryFilter
from .models import Library, Tag
from .serializers import LibrarySerializer, TagSerializer


class TagViewSet(viewsets.ModelViewSet):
    queryset = Tag.objects.all()
    serializer_class = TagSerializer


class LibraryViewSet(viewsets.ModelViewSet):
    queryset = Library.objects.all()
    serializer_class = LibrarySerializer
    filter_backends = (filters.DjangoFilterBackend,)
    filter_class = LibraryFilter

Using our API

You can now list all tags:

$ curl http://127.0.0.1:8000/tags/

Here is how result will look like (if you've created some data before):

[
  {
   "text": "python",
    "libraries": [
      "django",
      "django-rest-framework"
    ],
    "id": 1
  },
  {
    "text": "web",
    "libraries": [
      "django",
    ],
    "id": 2
  }
]

If you curl libraries:

$ curl http://127.0.0.1:8000/libraries/

your result will look like this:

[
  {
    "description": "cool web framework",
    "name": "django",
    "url": "https://www.djangoproject.com/",
    "tags": [
      "web",
      "python"
    ],
    "id": 1
  },
  {
    "description": "api framework",
    "name": "django-rest-framework",
    "url": "http://www.django-rest-framework.org/",
    "tags": [
      "python"
    ],
    "id": 2
  }
]

You can now perform queries with filters like below!

$ curl http://127.0.0.1:8000/libraries/?name=rest&tags=python

[
  {
    "description": "api framework",
    "name": "django-rest-framework",
    "url": "http://www.django-rest-framework.org/",
    "tags": [
      "python"
    ],
    "id": 2
  }
]

Pretty neat!

Summary

I hope that you'll find this blog post useful. Building an APIs can actually be a pretty complex task, because there are so many components that have to be connected.

Also, we've just scratched a surface. Our API doesn't have proper authorization, paging, ordering, permission system or tests, we also have pretty scarce validation, no api versioning and we only handle JSON.

You can download or read full code on github.