self-testing documentation

I wrote a Python module for making self-testing HTML and XHTML documentation: htmlselftest.py. "Self-testing" means that the procedure for verifying that the documentation is correct is embedded within the documentation itself. I'm sure someone has done this kind of thing before, but I was unable to find it after half an hour of searching Google -- so I wrote my own.

Here's a simple example of what a self-verifying document looks like:

<html>
<head><title>Hello world!</title></head>
<body>
<p>Type this command:
<code><test:prog id="hello">echo hello world</test:prog></code>.
The command should print
<code><test:read prog="hello">hello world
</test:read></code> back at you.
<test:prog_exit prog="hello"/>
</body>
</html>

The <test:xxx> tags are ignored by web browsers, so the tags are effectively invisible to the end-user.

[Update: Doug Landauer pointed out that doctest is the name of a standard python module, so I renamed it to htmlselftest.]

Posted on August 24, 2004 12:38 PM
More testing articles

Comments

So you wrote a test for comments too? :) Seriously, htmlselftest is pretty neat. It reminds me of a type of xUnit. Could you give an example of a typical error this type of testing might catch in an HTML document?

Posted by: Bill Glover at August 24, 2004 05:34 PM

An example would be documentation that says "run someprogram -o outfile.txt". If the -o flag changes to --output, then the documentation will be left telling the user the wrong thing.

Or maybe the code changes so that -o no longer works, even though it's supposed to. Then the documentation is technically correct, but the user will be annoyed because clearly you're so careless that you didn't even bother running the examples.

Maybe I should rename it to docUnit ;)

Posted by: Kim at August 24, 2004 06:03 PM