I think that everyone can agree that testing your project is very important.

I'm also omitting Test Driven Development controversials (Is TDD dead?). I like TDD, but don't use it everywhere for everything.

In this blogpost I want to focus on methods of testing.

So how and what do we test?

Testing styles

The most commonly known type of testing is unit testing.

If you've tested anything, you've probably used it.

Unit testing is a component of TDD - test driven development a pretty nifty development technique.

There are more styles out there: - BDD - behavioral driven development - randominzed testing (QuickCheck style) with randomized, parametrized tests

Unit testing

Unit testing is conceptually very simple. You focus on a simple unit of code, for example a function, prepare a simple setup and run the function with prepared arguments.

After that, you check results and side effects with assertions.

Example:

def test_spamming(self):
    person_to_spam = Person(name="John Tested")

    spam(person_to_spam)

    self.assertTrue(person_to_spam.was_spammed())

or

def test_rating_pizza(self):
    best_pizza = "Pizza with pepperoni"

    awful_pizza = "Pizza with fish"
    rating_of_best_pizza = rate(best_pizza)
    rating_of_awful_pizza = rate(awful_pizza)

    self.assertTrue(rating_of_best_pizza > rating_of_awful_pizza)
    self.assertEqual(rating_of_awful_pizza, 0)

Those examples are pretty simple, but should give you an idea how unit testing should look like.

Common concepts

There are some common concepts in unit testing:

• test fixture (setup)
• test case

test fixture (setup)

A test fixture represents the preparation needed to perform one or more tests, and any associate cleanup actions. This may involve, for example, creating temporary or proxy databases, directories, or starting a server process.

test case

A test case is the smallest unit of testing. It checks for a specific response to a particular set of inputs. unittest provides a base class, TestCase, which may be used to create new test cases.

test suite

A test suite is a collection of test cases, test suites, or both. It is used to aggregate tests that should be executed together.

mocking

Create an objects which mocks actual object and provides methods for additional validation, for example testing if expected arguments were passed to mock function and so on.

Mocks are mainly used to immitate objects which has lots of dependencies and isolate software under test from other code or external services.

You can read more about mocks on stackoverflow, on Martin Fowler blog or in python mock documentation.

Pros:

• tests if code executes properly
• tests your logic
• gives you confidence that your code won't broke in production

Cons:

• don't test whole behaviour
• it's easy to miss something
• takes time to write

How to do it in python?

Just use one of those great libraries:

• nose
• py.test
• unittest(2) (standardlib)

or if you are using django, default django testing library would do just fine.

Behavioral testing

Behavioral testing is a pretty neat idea. It takes much more 'bottom down' approach.

BDD is a second-generation, outside–in, pull-based, multiple-stakeholder, multiple-scale, high-automation, agile methodology. It describes a cycle of interactions with well-defined outputs, resulting in the delivery of working, tested software that matters.

BDD focuses on obtaining a clear understanding of desired software behavior through discussion with stakeholders. It extends TDD by writing test cases in a natural language that non-programmers can read. Behavior-driven developers use their native language in combination with the ubiquitous language of domain-driven design to describe the purpose and benefit of their code. This allows the developers to focus on why the code should be created, rather than the technical details, and minimizes translation between the technical language in which the code is written and the domain language spoken by the business, users, stakeholders, project management, etc.

Great sales pitch, but how does it look in real life? I have to say, that I haven't tried it in real life yet.

In python you can do behavioral testing using:

• lettuce
• behave

Lettuce is inspired by ruby cucumber and behave looks a bit more pythonic and better documented.

Grab a tutorial from behave.

Randomized testing

I love idea of randomized testing. It's amazing. I've first encountered this methodology while learning scala. Scala has a scalacheck library, which is a port of haskell's quickcheck.

After all I abandoned scala and moved to developing in python, but this idea of testing was ingrained into my brain.

So, what's going on here? And how do I do it in python?

Use pytest-quickcheck which is a plugin to excellent pytest.

Examples from pytest-quickcheck website:

@pytest.mark.randomize(i1=int, i2=int, ncalls=1)
def test_generate_ints(i1, i2):
    pass

Other resources about quickcheckstyle:

• python qc library
• post about quickcheck
• nose parametrized plugin

Functional/integration testing

Sometimes we have to put all the things together and test if they work. Of course we can do it manually, but it would be repetitive and boring.

People tend to avoid boring tasks, however no one likes getting alerts from production that everything is broken.

The best way to avoid such situations is to include integrational, functional testing into your workflow.

Functional testing can give you answers for:

• Is my app running?
• Is this user action displays correct view?
• Can I authorize in this API and for my query?

If you think about unit test as of lego building blocks, you can think compare functional testing to testing if those building blocks fit together and build you a robot who can run and do fun stuff.

They're usually much more complicated that unit tests and use more resources, take more time and so on, because there is no mocking there.

How can we automate functional testing in python. I know two functional testing libraries in python:

• python-selenium
• ghost.py

Summary

All of testing methodologies mentioned above are very useful. It depends on a project which one would be the best fit for you.

BDD tests would be great for app, but not so useful for a robust library and so on.

However, when you know what is available you can create your own mix, wich would work best for your particular use case.

That was a pretty long overview, however I feel that there is lots more to say about testing. In my next post I want to talk about good and bad testing (focusing on unit testing) and learning resources. Stay tuned!