How to write literate doctests with PyLit

Python Doctest Module

Doctests are a literate way of testing a Python script. They are supported by the doctest block syntax in reStructuredText.

The doctest module searches strings for pieces of text that look like interactive Python sessions, and then executes those sessions to verify that they work exactly as shown.

There are several common ways to use doctest:

• To check that a module’s docstrings are up-to-date by verifying that all interactive examples still work as documented.
• To perform regression testing by verifying that interactive examples from a test file or a test object work as expected.
• To write tutorial documentation for a package, liberally illustrated with input-output examples. Depending on whether the examples or the expository text are emphasised, this has the flavor of “literate testing” or “executable documentation”.

—Python Library Reference for the doctest module

Doctest and PyLit

The most common way to use the doctest module is to check examples in all docstrings of a module with doctest.testmod(), e.g.

if __name__ == "__main__":
    import doctest
    doctest.testmod()

You can also check a text file as if it were a docstring by calling doctest from the command line, e.g.

sh> python3 -c "import doctest; doctest.testfile('example.py.txt')"

Or, even simpler:

sh> python3 -m doctest example.py.txt

However, none of these methods will not check doctest blocks in comments. This is why they will fail to find doctests in the text blocks of a literate source in code format. (See the tutorial for discussion.)

You can of course convert your source to text form and run doctest.testfile on it. To simplify the task, Pylit supports Python doctests in a literate source with an option:

sh> python3 -m pylit --doctest example.py

will check a literate source file for all doctests regardless of their location in docstrings or text parts. It can work with both, text or code format.

In order to do this, it will read the file, transform a code source to text format on-the-fly and feed the result to a DocTestParser object. I.e., no text source file will be created if python3 -m pylit --doctest is called on a code source file.

This way, it is possible to separate basic examples in doc strings from additional test in the literate source.

Examples

testmod_literate

is a “literate version” of the example in the doctest module doc that does a self test when called as __main__.

It calls pylit.run_doctest to find tests in both docstrings and documentation blocks.

Test this file with:

#> python3 testmod_literate.py
0 failures in 11 tests

Sources: testmod_literate.py, testmod_literate.py.txt

testfile_literate

is a “literate version” of the example in the doctest module doc adapted for being tested with pylit --doctest.

Test this file with:

 MacBookPro-SLott:examples slott$ python3 pylit.py --doctest testfile_literate.py
 pylit.py:1644: UserWarning: data encoding=UTF-8 != docencoding=iso-8859-1
   warnings.warn( "data encoding={0} != docencoding={1}".format(data.encoding, docencoding) )
 0 failures in 19 tests

It imports itself in the usage example and has a non-testing (albeit
silly) default action if called from the command line.

Note the warning for a confused data encoding string. This is mostly
irrelevant, since most modern OS's don't really need the encoding hints any more.

Sources: `<testfile_literate.py>`_, `<testfile_literate.py.txt>`_