25.2. doctest — Test interactive Python examples¶
The doctest module searches for pieces of text that look like interactive
Python sessions, and then executes those sessions to verify that they work
exactly as shown. There are several common ways to use doctest:
To check that a module’s docstrings are up-to-date by verifying that all interactive examples still work as documented.
To perform regression testing by verifying that interactive examples from a test file or a test object work as expected.
To write tutorial documentation for a package, liberally illustrated with input-output examples. Depending on whether the examples or the expository text are emphasized, this has the flavor of “literate testing” or “executable documentation”.
Here’s a complete but small example module:
"""
This is the "example" module.
The example module supplies one function, factorial(). For example,
>>> factorial(5)
120
"""
def factorial(n):
"""Return the factorial of n, an exact integer >= 0.
If the result is small enough to fit in an int, return an int.
Else return a long.
>>> [factorial(n) for n in range(6)]
[1, 1, 2, 6, 24, 120]
>>> [factorial(long(n)) for n in range(6)]
[1, 1, 2, 6, 24, 120]
>>> factorial(30)
265252859812191058636308480000000L
>>> factorial(30L)
265252859812191058636308480000000L
>>> factorial(-1)
Traceback (most recent call last):
...
ValueError: n must be >= 0
Factorials of floats are OK, but the float must be an exact integer:
>>> factorial(30.1)
Traceback (most recent call last):
...
ValueError: n must be exact integer
>>> factorial(30.0)
265252859812191058636308480000000L
It must also not be ridiculously large:
>>> factorial(1e100)
Traceback (most recent call last):
...
OverflowError: n too large
"""
import math
if not n >= 0:
raise ValueError("n must be >= 0")
if math.floor(n) != n:
raise ValueError("n must be exact integer")
if n+1 == n: # catch a value like 1e300
raise OverflowError("n too large")
result = 1
factor = 2
while factor <= n:
result *= factor
factor += 1
return result
if __name__ == "__main__":
import doctest
doctest.testmod()
If you run example.py directly from the command line, doctest
works its magic:
$ python example.py
$
There’s no output! That’s normal, and it means all the examples worked. Pass
-v to the script, and doctest prints a detailed log of what
it’s trying, and prints a summary at the end:
$ python example.py -v
Trying:
factorial(5)
Expecting:
120
ok
Trying:
[factorial(n) for n in range(6)]
Expecting:
[1, 1, 2, 6, 24, 120]
ok
Trying:
[factorial(long(n)) for n in range(6)]
Expecting:
[1, 1, 2, 6, 24, 120]
ok
And so on, eventually ending with:
Trying:
factorial(1e100)
Expecting:
Traceback (most recent call last):
...
OverflowError: n too large
ok
2 items passed all tests:
1 tests in __main__
8 tests in __main__.factorial
9 tests in 2 items.
9 passed and 0 failed.
Test passed.
$
That’s all you need to know to start making productive use of doctest!
Jump in. The following sections provide full details. Note that there are many
examples of doctests in the standard Python test suite and libraries.
Especially useful examples can be found in the standard test file
Lib/test/test_doctest.py.
25.2.1. Simple Usage: Checking Examples in Docstrings¶
The simplest way to start using doctest (but not necessarily the way you’ll
continue to do it) is to end each module M with:
if __name__ == "__main__":
import doctest
doctest.testmod()
doctest then examines docstrings in module M.
Running the module as a script causes the examples in the docstrings to get executed and verified:
python M.py
This won’t display anything unless an example fails, in which case the failing
example(s) and the cause(s) of the failure(s) are printed to stdout, and the
final line of output is ***Test Failed*** N failures., where N is the
number of examples that failed.
Run it with the -v switch instead:
python M.py -v
and a detailed report of all examples tried is printed to standard output, along with assorted summaries at the end.
You can force verbose mode by passing verbose=True to testmod(), or
prohibit it by passing verbose=False. In either of those cases,
sys.argv is not examined by testmod() (so passing -v or not
has no effect).
Since Python 2.6, there is also a command line shortcut for running
testmod(). You can instruct the Python interpreter to run the doctest
module directly from the standard library and pass the module name(s) on the
command line:
python -m doctest -v example.py
This will import example.py as a standalone module and run
testmod() on it. Note that this may not work correctly if the file is
part of a package and imports other submodules from that package.
25.2.2. Simple Usage: Checking Examples in a Text File¶
Another simple application of doctest is testing interactive examples in a text
file. This can be done with the testfile() function:
import doctest
doctest.testfile("example.txt")
That short script executes and verifies any interactive Python examples
contained in the file example.txt. The file content is treated as if it
were a single giant docstring; the file doesn’t need to contain a Python
program! For example, perhaps example.txt contains this:
The ``example`` module
======================
Using ``factorial``
-------------------
This is an example text file in reStructuredText format. First import
``factorial`` from the ``example`` module:
>>> from example import factorial
Now use it:
>>> factorial(6)
120
Running doctest.testfile("example.txt") then finds the error in this
documentation:
File "./example.txt", line 14, in example.txt
Failed example:
factorial(6)
Expected:
120
Got:
720
As with testmod(), testfile() won’t display anything unless an
example fails. If an example does fail, then the failing example(s) and the
cause(s) of the failure(s) are printed to stdout, using the same format as
testmod().
By default, testfile() looks for files in the calling module’s directory.
See section Basic API for a description of the optional arguments
that can be used to tell it to look for files in other locations.
Like testmod(), testfile()’s verbosity can be set with the
-v command-line switch or with the optional keyword argument
verbose.
Since Python 2.6, there is also a command line shortcut for running
testfile(). You can instruct the Python interpreter to run the doctest
module directly from the standard library and pass the file name(s) on the
command line:
python -m doctest -v example.txt
Because the file name does not end with .py, doctest infers that
it must be run with testfile(), not testmod().
For more information on testfile(), see section Basic API.
25.2.3. How It Works¶
This section examines in detail how doctest works: which docstrings it looks at, how it finds interactive examples, what execution context it uses, how it handles exceptions, and how option flags can be used to control its behavior. This is the information that you need to know to write doctest examples; for information about actually running doctest on these examples, see the following sections.
25.2.3.1. Which Docstrings Are Examined?¶
The module docstring, and all function, class and method docstrings are searched. Objects imported into the module are not searched.
In addition, if M.__test__ exists and “is true”, it must be a dict, and each
entry maps a (string) name to a function object, class object, or string.
Function and class object docstrings found from M.__test__ are searched, and
strings are treated as if they were docstrings. In output, a key K in
M.__test__ appears with name
<name of M>.__test__.K
Any classes found are recursively searched similarly, to test docstrings in their contained methods and nested classes.
Changed in version 2.4: A “private name” concept is deprecated and no longer documented.
25.2.3.2. How are Docstring Examples Recognized?¶
In most cases a copy-and-paste of an interactive console session works fine, but doctest isn’t trying to do an exact emulation of any specific Python shell.
>>> # comments are ignored
>>> x = 12
>>> x
12
>>> if x ==