Python and doctest

I've just recently got to learn Python, and after a bit of struggling with the new syntax, I found the language to be very appealing. It's well structured, and has been around for a long time so if you can't find the library you are looking for in pythons standard libraries (which is very unlikely), there are a bunch of other libraries to install (for example lxml, which is a great xml/xpath wrapper for libxml2). The only issue I've had so far is with some libraries not being compatible with python 3.1 (which differs a lot from 2.x), but this is probably a transition period soon being over.

And as with all languages, sooner or later you would like to know how to setup your tests. And when googling "python test framework", I stumbled upon doctest:

The doctest module searches for pieces of text that look like interactive Python sessions, and then executes those sessions to verify that they work exactly as shown.

This is something I've never seen before, but yet strikes me as very powerful. To write your tests, you simply include an example execution of the function you want to test in the comment, i.e.:

'''
This function prints the string "Hello World!".

For example:
>>> say_hello_world()
Hello World!
'''
def say_hello_world():
print("Hello World!")

To run the tests, you can either include the test execution in the main method of the script file, which to me is wrong if you would like to have another default main method. Another way to do it (if you have python 2.6 or later) is to execute like this:

python -m doctest -v helloworld.py

This will load the doctest module, and then execute the tests in helloworld.py, which will call the function say_hello_world(), and make sure it's output equals "Hello World!", as written in the comment. You then extend the tests by simply including more examples in your comment.

IMO, the real power of this is that you don't need to write separate test functions, plus you get the documentation for free. I.e., with xunit the same test would look something like this:

def test_say_hello_world():
mocked_print = create_print_mock_function()
say_hello_world()
xunit.assert_equal("Hello World!", mocked_print.get())

And I'm not even sure if create_print_mock_function is possible to do. Even if it would, the tests becomes much larger, and harder to maintain.

The main downside I see to using doctest is that it depends on the expected output being formatted exactly as python would have formatted it. For example, the following function will fail, just because python will include extra spaces in the array:

'''
>>> get_array()
[1,2]
'''
def get_array():
return [1,2]

will fail with

Failed example:
get_array()
Expected:
[1,2]
Got:
[1, 2]

In this case it's easy to fix, but how will it scale when you have hundreds of tests? What happens when you update your tostring methods? Instead of inspecting the returned data, you inspect the data formatted to a string, which can very well mean something quite different if you are writing your own tostring methods.

Anyway, at the latest coding dojo (Majorna Coding Dojo!), we decided to give python and doctest a try, and it worked very well. Even though most of us didn't know Python from before, we didn't have any problems using doctest.

I will definitely use doctest more in the future, and will even see if I can find similar testing frameworks (Document Driven Development, DDD) for other languages such as Ruby, C# and C++. So if you are into Python programming, I definitely recommend you trying this out.