Tutorial¶

How to Write Literate Programs with PyLit

Hello World¶

We start with a classic example in Python

# The classical programming example in Python3

print( "Hello world." )

save it as hello.py and convert to a reStructured Text document with pylit.py:

#> python3 pylit.py hello.py
extract written to hello.py.txt

The output file hello.py.txt looks like

The classical programming example in Python3

::

  print( "Hello world." )

We can see the difference between “commented code” and “code living in a text document”.

Points to mention:

One can start literate programming with an existing code file (and without knowledge of reStructured Text syntax).
Documentation is uncommented (if it is separated from code by a blank line and has a recognised comment string at the start of each line).
A double colon is added at the end of the text block. It is the reStructured Text marker for the following literal block. (No marker is added, if the text block already ends with a double colon.)
Code is indented to form a literal block. It will be printed using a monospaced font and without reStructured Text substitutions.
PyLit adds “.txt” to the filename for the text version.

Now we can add some more documentation and a link (of course, knowledge of reStructured Text syntax helps in this stage):

The classical programming example in Python3

Let Python_ print a string literal::

  print( "Hello world." )

It's easy, isn't it?

.. _Python: www.python.org

Pretty-printed with Docutils, it looks like

Example Output

The classical programming example in Python3

Let Python print a string literal:

print( "Hello world." )

It’s easy, isn’t it?

If we re-convert the result to code,

#> python3 pylit.py hello_2.py.txt
extract written to hello_2.py

we get

# The classical programming example in Python3
#
# Let Python_ print a string literal::

print( "Hello world." )

# It's easy, isn't it?
#
# .. _Python: www.python.org

Points to mention:

The double colon that was added in the first conversion is not stripped in the re-conversion.

(Generally, a round-trip should not introduce changes after the first cycle. This way it is ensured that the line-numbers are the same in text and code source.)
The code block ends at the first non indented line (Precisely, at the first line that is not more indented than the preceding text block.)

Text Blocks and Comments¶

Comment lines are only transformed to a text block, if they

start with a matching comment string (whitespace counts!, the Python default is '# '), and
are separated from non-text lines by at least one blank [1] line

Otherwise, they are kept as commented code.

An example will illustrate this. The code:

# 99bottles.py -- print the famous "99 bottles of beer" song lyrics

# Introductory example to literate programming
#
# count down from 99 to 1
for bottles in range(99,0,-1):
    ....

is mapped to text as:

99bottles.py -- print the famous "99 bottles of beer" song lyrics

::

  # Introductory example to literate programming
  #
  # count down from 99 to 1
  for bottles in range(99,0,-1):
      ....

The comment in the 5th line marks the “secondary documentation” as part of the code block.

However,

# 99bottles.py -- print the famous "99 bottles of beer" song lyrics
#
# Introductory example to literate programming

# count down from 99 to 1
for bottles in range(99,0,-1):
    ....

is mapped to text as:

99bottles.py -- print the famous "99 bottles of beer" song lyrics

Introductory example to literate programming

::

  # count down from 99 to 1
  for bottles in range(99,0,-1):
      ....

The comment in the 2nd line is removed, as it is inside a documentation block.

[1]	a line is considered blank, if it contains only whitespace

Ordinary Literal Blocks¶

How can I include a literal block that should not be in the executable code (e.g. an example, an earlier version or variant)?

Workarounds:

Python session examples and doctests can use doctest block syntax. See the doctests section.
Use a “code-block” directive and set the code_block_marker option or use a distinct directive for ordinary literal blocks.

Drawback: such directives are not part of the core rst syntax (yet) but must be defined in an add-on (see syntax highlight for examples).

Use a parsed-literal block directive and mark lines containing “accidental” markup as inline literals. E.g. the text

This will be printed as literal block
but not become part of the source code:

.. parsed-literal::

  print( "code example that should not run" )
  ``result = 5 *n* 2``

will be typeset as

This will be printed as literal block but not become part of the source code:
print( "code example that should not run" )
result = 5 *n* 2

File Headers¶

Sometimes code needs to remain on the first line(s) of the document to be valid. The most common example is the shebang line that tells a POSIX shell how to process an executable file. In Python, the magic comment specifying the source code encoding must occur on line one or two:

#!/usr/bin/env python3
# -*- coding: iso-8859-1 -*-

# The classical programming example in Python3::

print( "Hello world." )

Headers are converted to a comment in the text source:

..  #!/usr/bin/env python3
  # -*- coding: iso-8859-1 -*-

The classical programming example in Python3::

  print( "Hello world." )

Pretty-printed with Docutils, it looks like

hello_with_header

The classical programming example in Python3:

print( "Hello world." )

Everything before the first text block (i.e. before the first paragraph using the matching comment string) will be hidden in HTML or PDF output.

It may come as surprise that a part of the file is not “printed”. (In the case that there is no matching comment at all, the complete code source will become a comment, however, in this case it is not likely the source is a literate program anyway). But there are advantages also:

line numbers are kept during the text <–> code conversion (which would be impossible with a literal block marker as this needs to be at the end of the preceding paragraph)
you can hide a repeating (or boring) header in a project consisting of many source files.

If needed for the documentation, it is possible to repeat the header in the the first text block, e.g. using a parsed literal block:

#!/usr/bin/env python3
# -- coding: iso-8859-1 --

Doctests¶

Pylit supports Python doctests in a literate script.

We add a doctest block [2] to our example:

# The classical programming example in Python will print a familiar greeting:
#
# >>> with open("docs/tutorial/hello_with_doctest.py") as source:
# ...     exec( compile( source.read(), source.name, 'exec' ) )
# Hello world.
#
# ::

print( "Hello world." )

Now try it with

#>  python -c "import doctest; doctest.testfile('hello_with_doctest.py')"

There is no output. So everything is OK? Unfortunately not: doctest.testfile does not find the test, as it is “hidden” in a comment. [3]

Pylit converts the source to the text version, feeds it to the doctest module’s Advanced API (introduced in Python 2.4), and we get

#> pylit --doctest hello_with_doctest.py
**********************************************************************
File "hello_with_doctest.py", line 3, in
Failed example:
    execfile("hello_with_doctest.py")
Expected:
    Hello world
Got:
    Hello world.

Ah yes, we forgot the full-stop in our test. Adding it and testing again:

#> pylit --doctest hello_with_doctest_2.py
0 failures in 1 tests

The printed summary will ensure us that the test actually passed.

Read more about doctests in the literate doctests example.

[2]	There is no double colon before the doctest; a doctest block is recognised by starting with the Python interpreter prompt `>>>` instead.

[3]	The tests will be found, if `doctest.testfile` is run on the text source, i.e. `python -c "import doctest; doctest.testfile('hello_with_doctest.py.txt')"`

Including Files¶

PyLit does not allow the specification of a separate output file for individual code blocks like e.g. noweb. The “dual source” concept limits the choice to one output file per input file. However, this can be compensated by the use of the include directive.

Let us assume that for some reason, the friendly greeting should be defined in a separate file greeting.py:

# a friendly greeting stored in a variable ::

greeting = "Hello world."

The documentation of the calling file can include the executed file

# The classical programming example with the greeting from a variable in
# another file::

import greeting
print( greeting.greeting )

# Where ``greeting.py`` contains:
#
# .. include:: greeting.py.txt

Saved to hello_multifile.py.txt and pretty-printed with Docutils, this looks like

hello_multifile

The classical programming example with the greeting from a variable in another file:

import greeting
print( greeting.greeting )

Where greeting.py contains:

a friendly greeting stored in a variable

greeting = "Hello world."

you have to convert both, greeting.py and hello_multifile.py. (Currently, pylit cannot do ‘batch processing’ of multiple input files.)

Tutorial¶

Hello World¶

Text Blocks and Comments¶

Ordinary Literal Blocks¶

File Headers¶

Doctests¶

Including Files¶

PyLit

Navigation

Related Topics