Using programmer tests in place of some or all documentation?

I’m writing an article for an agile software magazine about how to use programmer tests in place of some or all written documentation, and I’d appreciate some feedback on a very early outline for the article. This is what I intend to cover:


  1. Different audiences for documentation


    1. Application programmers — users of our libraries

    2. Maintenance programmers — modifiers and extenders of our libraries


  2. Components of a good test


    1. Good names

    2. Clearly defined setup, processing, and asserting sections


  3. Test sufficiency — do I tell the whole story with my tests


    1. Capturing design decisions

    2. Covering interesting use cases

    3. Showing error behavior

    4. Interactions with the environment

    5. Test organization for different audiences


      1. Application Programmers want to find common scenarios easily

      2. Maintenance Programmers want to follow original developer’s train of thought



  4. What prose and UML are needed to supplement these tests

  5. Future directions in Tests as Documentation (Automation topics mostly)

Obviously this is still pretty rough. I intend to use the tests for a Caching block I just wrote for Microsoft Enterprise Library along with pseudocode for the library to show the design decisions. The goal is to show devs how to create tests that will server as documentation.

Does this sound remotely interesting? Any suggestions about other things to cover?

And for those of you in St. Louis, I’m going to be giving a talk on this subject at this month’s XPSTL (9/1).

— bab

8 thoughts to “Using programmer tests in place of some or all documentation?”

  1. Brian,

    The TOC you’ve put up looks interesting and I would be keen to see the article. I recently blogged about Agile Methodologies and Documentation here (http://dotnetjunkies.com/WebLog/netblog/archive/2004/07/17/19566.aspx)

    I am working on an XP project now. Some of my colleagues believe that tests can serve as the documentation and specification. I do NOT agree with this. I fail to see how a test can capture design decisions for a class much less an entire project. I have tried to understand code bases when all that was provided was tests and chucked them out in frustration. Where do you start? How do you know what the system is supposed to do overall before delving into the internals?

    Business Requirements cannot be captured by tests or understood by non technical users which is why UML use cases are so popular for capturing requirements. Software Architecture is not documented by programmer tests but by established Conceptual, Logical, Physical and Deployment Views.

    I am using TDD and am happy with the benefits it provides me as a developer. I like having a battery of tests to run against the code to show it works. But my clients will not accept the tests as proof that all requirements were captured (ie) where are the requirements documented?

    I was reading the TDD in .NET book recently and the authors show the example of a stack and proceed to list all the requirements for a stack in the process of explaining how ‘specifications’ are done and proved with tests.

    IMHO, this is a rather trivial example . These are low level requirements for a class and are programmer specs , not functional requirements for an entire component

    Anyway, for my part, I plan to supplement my working code with a proper Software Architecture document and the corresponding Tech Architecture and Deployment Specs.

    But I like learning new stuff (which is why I have latched on to TDD happily) and it would be cool to see if it is indeed possible to make tests serve as goofd documentation.

    Cheers

    Benjy

  2. I think it would be hard to show your design decisions in code, but it looks like you are spending the bulk of the article answer whether that is possible or not. I’m looking forward to it.

  3. I think this sounds interesting as well. I really don’t see how you can show the design decisions in your unit tests without making them into something they aren’t.

  4. I think it sounds very interesting. New concept to me – so my mind is open to all ideas.

    Please post the article on your blog aswell.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.