Bit rot in the documentation

This week I’m attending Web Directions South 2014, a conference for people who “design, imagine, create or build digital products, web sites and applications”. It’s a privilege to be here, and to speak at this cool get-together.

My session is called Bit Rot in the Docs. (The link goes to the slides on SlideShare.) The presentation looks at how documentation can degenerate over time, and some techniques for finding and fixing errors.

The very mention of bit rot will strike terror into an engineer’s heart. “Unused programs or features will often stop working after sufficient time has passed, even if ‘nothing has changed’.” (From the hacker’s Jargon File.)

Bit rot affects documentation too. Whether the cause be a changing environment, human error, or the infamous cosmic rays, we need to root out that rot. But should the technical writer read and test the documentation on a regular basis? That’s the least efficient and most error-prone way of detecting doc decay.

Instead, in this session we looked at the following techniques:

  • Automated testing of code samples.
  • Documentation reviews as a standard part of engineering team procedure.
  • Collaborative spot-testing.
  • Customer feedback.

For more details of the session, see the slides on SlideShare.

Bit Rot in the Docs

About Sarah Maddox

Technical writer, author and blogger in Sydney

Posted on 30 October 2014, in technical writing and tagged , , , , , . Bookmark the permalink. 6 Comments.

  1. Thanks for bring this topic up, Sarah. The problem is likely to become more acute as we increasingly use reviewing workflows that are topic-based or that focus on only the content that’s new and changed. It’s also aggravated by turnover on the documentation and development teams, where neither the technical writer nor the SME might have any idea what was put into the docs just a year or two ago.

    I like your ideas of collaborative spot testing and automated testing of code samples. Of course we’ll have to justify the expense of each one, even if the expense is relatively small. Perhaps the key is to improve the customer feedback loop so that everyone can see how bit rot in the docs results in angry customers and/or preventable calls to Tech Support.

    Definitely this is something that we, the Tech Comm community, should talk about.

    • Hallo Larry

      Great comment! I think you’re right about the customer feedback loop. The biggest thing I’ve learned from giving people an opportunity to give feedback on the docs, is that they will do it! They’ll tell you when things have gone squiffy, and they’ll suggest improvements.

      Another powerful technique is to get the whole team involved in the docs. When the engineers know what’s in the docs, and feel ownership of the docs, they’ll feel the need to keep the docs shiny.

      I recently ran a doc bug bash, where engineers as well as tech writers focused on fixing doc errors for two days. We dubbed one of the engineers an “honorary tech writer” due to the quality and thoroughness of his fixes!


  2. Rich declarative markup can do a huge amount to detect bit rot in the docs. If we simply markup up references to significant subjects to declare exactly what they are referring to, and then validate that appropriate resources exist to explain those subjects, we can detect all kinds of orphaned subject matter.

    Incidentally, we can use the same markup to automatically generate links, eliminating link management and link rot as concerns at the same time.

    One of my ongoing rants is this: one of the most valuable uses of structured content is validation. The validation potential is huge, and largely ignored. As the size of our information sets grows, and the rate of change accelerates, bit rot will accelerate unless we take major steps to improve the validation capability of our content.

    • I think you’re right, Mark. I was a book indexer for a few years. One of the books I indexed was an annual publication of 900+ pages. When I started in the role, we were preparing the index separately from the book. The next year, I suggested that we swap to embedded indexing – I added index tags directly into the content, using Quark XPress. This was arduous the first time, but made subsequent editions much easier to index and ensured the index references were more likely to remain up to date.

      One thing to note is that the editor and I had to work closely together, to ensure that the tags and content didn’t get out of sync. For example, if the editor adapted a paragraph or moved content around without taking the index tags into account, the references would be incorrect. She might remove a reference to something, without removing the associated tag. Or she might move a block of content from one place to another, without moving the tag.

      In the same way, technical writers and other people updating the docs would need to take the source references into account.

      But that’d be a small price to pay, and it’s the kind of thing we’re good at. 🙂

  3. Hi Sarah, this is a great idea for a presentation. I’m interested to know how you do automated testing of code samples, as mentioned in your slides. Do you write tests when you write code samples, then run nightly builds? What tools do you use for this?

    • Hallo Andrew 🙂

      The source files for the JavaScript samples are in the same code repository as the API, and are hooked into the automated build system. It’s a continuous integration system, which automatically runs the builds as soon as anyone submits a change to the repo. If a build is broken, it sends an email notification to a particular group.

      We use an internal toolset, and the engineers write the tests. Setting up a continuous integration server and automated testing is non-trivial. I recommend that the documentation uses the same repo and tools as the engineering team uses.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: