Writing a book with DocBook and a Confluence wiki

We’re in the final stages before sending my book off to the printers. Exciting! While we wait, let me tell you a bit about how the publishing team and I have produced the book. We’re using a Confluence wiki and DocBook XML.

Here’s our process in brief:

• Plan, write and review the book on a Confluence site.
• Use the Scroll Wiki DocBook Exporter to convert the content to DocBook XML.
• Use DocBook XSL-FO style sheets to create a PDF file for sending to the printers.
• Use XSL to generate ebook formats too.

This post is about writing and reviewing the book on the wiki, and converting the Confluence content to DocBook XML – the first two points in the list above. Richard Hamilton, at XML Press, is the expert on the further DocBook processing.

A bit about the book

The book is called Confluence, Tech Comm, Chocolate: A wiki as platform extraordinaire for technical communication. Details are at XML Press. It is all about developing technical documentation on a wiki, with a focus on Confluence wiki. And just to be ultra meta, I’ve written the book on a Confluence site. Dogfooding FTW!

Writing, planning and technical review on the wiki

Richard Hamilton and I have spent the last nine months on a Confluence wiki site. We kicked off our planning there in May 2011. Since then, I’ve spent around 400 hours writing the content on the wiki. Ryan Maddox, our illustrator, has uploaded his images onto the wiki. For two weeks in early December, the technical reviewers joined us there too – six knowledgeable and enthusiastic people:

• Alan J. Porter – Writer, speaker, consultant
• Anne Gentle – Technical writer for OpenStack
• Ellen Feaheny – CEO of AppFusions
• Mark Fidelman – Social business GM at harmon.ie
• Robert Rhyne Armstrong – Director of documentation for RouteMatch Inc
• Sherif Mansour – Technical product manager at Atlassian

The wiki was abuzz with review comments, opinions and counter-opinions, laughter and chat.

Now it’s gone a bit quiet while Richard and I work on the last stages of book production. When we launch the book, we’ll open up the wiki site too. You’ll be able to join us there and make it buzz again. 🙂

DocBook for the publication processes

DocBook is an XML standard for documents, developed by O’Reilly as a means of making their publishing process more efficient. It is now an open standard maintained by OASIS.  Richard Hamilton, at XML Press, uses DocBook XML to publish a range of books on the subject of technical communication. Using a customised set of the open source XSL style sheets for DocBook,  Richard can create HTML, PDF (through XSL-FO), EPUB and other epublishing formats from the DocBook source.

Converting content from Confluence wiki to DocBook XML

So, I’ve finished writing the book and the reviewers have worked their magic too. It’s all on a Confluence wiki. The content is stored in the Confluence database in wiki format. How do we get it to DocBook so that Richard can create the print and ebook formats?

Enter the Scroll Wiki DocBook Exporter.

Scroll Wiki DocBook Exporter is a plugin for Confluence, developed by K15t Software. (A plugin is a small piece of software that extends the functionality of the wiki. It is similar to an add-on for a web browser.) Once you have installed the Scroll Wiki DocBook Exporter on your Confluence site, you can export a page, a set of pages or an entire space, to DocBook XML.

How to use the Scroll Wiki DocBook Exporter

The first thing is to add the plugin to your Confluence site. You need to be a Confluence system administrator, then you can install the plugin in the usual way:

1. Log in as a Confluence system administrator.
2. Choose “Browse” > “Confluence Admin” > “Plugins” > “Install”.
3. Type “scroll wiki docbook exporter” into the search box and click “Search”.
4. Click the name of the plugin in the list of search results, to open the panel showing the plugin details.
5. Click “Install Now”.

See the Confluence documentation on installing plugins.

You will need a license key from K15t Software. They provide a free evaluation license that gives you full functionality for 30 days.

Once the plugin is installed, a new option appears in the Confluence “Tools” menu, allowing you to export the content to DocBook format.

1. Go to the page that you want to export. If you want to export a set of pages, go to the parent page of the set.
2. Click “Tools” > “Export to DocBook”.
3. Adjust the options in the dialog that pops up:

Scroll Wiki DocBook Exporter

When you are ready, click “Start Export”. The plugin will create a zipped file containing the DocBook XML and attachments, and will offer the file to you for downloading.

A sample of the output

Here is one of the book’s pages on the wiki:

Front page of the book on the wiki

And here is an extract of the DocBook XML:

<?xml version=”1.0″ encoding=”UTF-8″?>
<book xmlns=”http://docbook.org/ns/docbook&#8221; xmlns:xlink=”http://www.w3.org/1999/xlink&#8221; xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance&#8221; xsi:schemaLocation=”http://docbook.org/ns/docbook http://docbook.org/xml/5.0/xsd/docbook.xsd&#8221; version=”5.0″ xml:id=”scroll-bookmark-1″>
<info>
<title>Confluence, Tech Comm, Chocolate</title>
</info>
<part xml:id=”scroll-bookmark-2″>
<title>Confluence, Tech Comm, Chocolate</title>
<chapter xml:id=”scroll-bookmark-3″>
<title>A wiki as platform extraordinaire for technical communication</title>
<para>By Sarah Maddox</para>
</chapter>
</part>

[SNIP]

<index />
</book>

Index, footnotes and figure captions

The Scroll Wiki DocBook Exporter supports these too. The plugin adds a number of macros to Confluence, which you can use to mark up the index entries, footnotes and figure captions. I’ll write another post with the details. I’m sure many people are agog to know how this works. 😉

Working with K15t Software and XML Press

Richard and I have worked on this project with Tobias Anstett and the rest of the team K15t Software.  They are great people to work with: knowledgeable, enthusiastic and energetic. As far as I know, this is the first time anyone has written a book on Confluence for publication via DocBook XML. We have added new functionality to the plugin, especially for the index and footnotes, adapting and tuning as we go.

Thank you all. It’s exciting to help perfect a plugin that many other authors and technical writers will be able to use.

And I can’t wait to get my hands on a copy of the book!

REST API documentation embedded in the application

Our development team has built a tool that documents an application’s REST APIs within the application itself. What’s more, you can test the REST resources and methods too. All from the application’s user interface. Now, that’s embedded help for nerds. 🙂 I’m writing this post because I think many technical writers and developers will be interested in this solution. It may trigger ideas about adding something similar to other applications too.

The tool is called the REST API Browser, and it is implemented as a plugin. At the moment, it is available only within the Atlassian Plugin SDK. In the future, you may be able to download the REST API Browser as a separate plugin and install it into any Atlassian application.

So, what does the REST API Browser do, what is the Atlassian Plugin SDK, and how can you get the REST API Browser to work within an Atlassian application?

Overview of the REST API Browser

This is what the REST API Browser looks like, when running inside an application called JIRA:

The /user/search REST resource in JIRA

JIRA is an issue tracker developed by Atlassian. The above screenshot shows one of the JIRA administration screens, part of the application’s user interface. I’m running JIRA on my local machine, in plugin development mode. The REST API Browser is available as one of the options on the application’s administration screens. It’s as simple as that.

In the screenshot, you can see the resources in the JIRA REST API, starting at the /user/search resource. For each resource, the REST API Browser shows the methods (GET, POST, PUT) and the parameters available.

You can even do real-time testing of the APIs, by submitting a request and seeing the response. In the screenshot below, I have run a GET request using the /user/search resource, asking for details of username “admin”. You can see:

• A form, prompting you for the parameters relevant to the resource and method. In this case, the only parameter is the username.
• The request, as formulated by the REST API Browser.
• The response headers.
• The JSON in the response body.

A REST request and response

The application’s REST APIs at your fingertips!

Which REST APIs does the Browser show?

The REST API Browser displays all the REST and JSON-RPC APIs available in the running installation of the application. That means all the remote APIs that are part of the application’s default installation (JIRA, in this case) as well as any additional REST APIs provided by a plugin.

So, if you install a plugin into JIRA, and that plugin exposes a REST API, the resources will show up in the REST API Browser too. Magic.

Introduction to Atlassian Plugin SDK

The Atlassian Plugin SDK is a tool for developers who want to create a plugin (add-on) for an Atlassian application. For example, you may want to add a new option to the Confluence page menu, showing all authors who have ever updated the current page. Or you may want to add a new option in JIRA that points to your organization’s intranet site.

But the SDK is useful even for people who don’t want to build a plugin. You can use the SDK to download and run an Atlassian application on your own machine, in plugin developer mode. One of the features that you get is the REST API Browser.

How can you get hold of such sweetness?

Here’s a quick start guide. First, install the SDK and use it to download and run your application:

1. Follow the guide to installing the Atlassian Plugin SDK. Do just steps 1, 2 and 3. You can skip the last step, which sets up an IDE (Eclipse, IDEA or NetBeans), if you do not need a development environment.
2. Create a directory in your file system to store the application executables. Let’s assume you want to run the REST API Browser in JIRA. Then, for example, you could create a directory called myjira.
3. Open a command window.
4. Go to the new directory that you just created, myjira.
5. Run atlas-run-standalone --product APPLICATION. For example, atlas-run-standalone --product jira.
   Note: There are two dashes in front of the word product.

After following the above steps, you will have the application running on your computer. When all the downloads and installation are complete (which may take a while), you will see a message in your command window that includes the URL for the local installation of the application.

For JIRA, the message will look something like this:

[INFO] jira started successfully in 174s at http://localhost:2990/jira
[INFO] Type CTRL-D to shutdown gracefully
[INFO] Type CTRL-C to exit

Now you can access the application in your browser and use the REST API Browser from the application user interface:

1. Go to your web browser and enter the URL given for your application. For example, http://localhost:2990/jira.
2. Enter the username and password: admin / admin.
3. Go to the application’s administration screen. For example, in JIRA: Click Administration at top right of the screen.
4. Click REST API Browser on the administration screen.
5. Choose an API from the dropdown list at the top left of the screen.
6. Choose a resource from the list on the left of the screen.
7. See the methods (GET, POST, PUT, etc) and the parameters available for that resource.
8. To test the resource, enter the parameter values as prompted then click Execute.

The documentation has the details of running the REST API Browser in the SDK, and of viewing and testing the resources.

Getting technical: How the REST API Browser works

The source code is available on Bitbucket, as part of the Atlassian Developer Toolbox. The developer’s guide describes how to ensure that a REST API is included in the Remote API Browser. That document also gives a summary of how the browser is put together.

Pretty neat, huh

From a plugin developer’s point of view, the REST API Browser is very useful. From a technical writer’s point of view, I think it’s pretty revolutionary. Has anyone seen other examples of embedded REST API documentation?

Kudos to Rich Manalang and the Atlassian developer relations team for developing this shiny tool. Here is the blog post in which Rich announced it: Introducing the REST API Browser and the Atlassian Developer Toolbox.

Scroll converts wiki to DocBook and PDF

K15t Software have just announced release 1.0 of Scroll, a Confluence plugin that converts wiki documentation to DocBook and PDF. I’ve been interested in this project since the early beta days and I’m excited for the K15t guys about the functionality Scroll offers already and about their plans for even more awesome developments.

First things first: You’re probably wondering why they’re called “K15t Software” 😉 It’s because one of the developers and company founders is Stefan Kleineikenscheidt. I guess that most people, like me, go cross-eyed upon encountering his surname. So he uses “K15t”, in the proud tradition of “i18n” and other such abbreviations. Another K15t Software developer is Tobias Anstett, fondly known as “t11t”. During Scroll’s beta period we’ve had some good conversations via Twitter, blogs, video conferencing and email.

History in the making

The guys founded K15t Software as a company just a few weeks ago. If you’re a technical writer, you’ll like their mission statement:

“K15t Software is a small development shop from Stuttgart, Germany. Its mission is to build useful software products with the focus on tools for wiki-based documentation.”

It all started five years ago, when Stefan lodged a request for a DocBook export from Confluence wiki. The request is on the Atlassian JIRA site — CONF-762. A DocBook exporter is not on the roadmap for core Confluence, so Stefan and the team decided to build one as a plugin. And the rest is history! Well, it’s the future too, because K15t plan to keep adding features to Scroll.

How does Scroll work?

It’s a plugin (add-on) to Confluence. If you have administrator permissions on your Confluence site, you can install the plugin as described in the Scroll Installation Guide.

Once it’s installed, the “Scroll Wiki Exporter” option will appear in your Confluence menu. Here’s a screenshot of the option in Confluence 2.10. I’m on a wiki page called the “Crowd User Guide” and have opened the “Tools” menu on the right:

Scroll converts wiki to DocBook and PDF

When you choose the “Scroll Wiki Exporter” option, you get a screen with some tabs where you can choose either PDF or DocBook output and set some formatting options. On the “Document Information” tab you can set the document title, author and version. Scroll will provide default values based on the wiki page, but you can change them:

Scroll converts wiki to DocBook and PDF

The “Page Tree” tab shows you a hierarchical view of the pages you are about to export. The top level of the tree is always the page you were on when you clicked the “Scroll Wiki Exporter” option. The export will include that page plus all its sub-pages and their sub-pages etc. At the moment, “Page Tree” tab is view only. Scroll may allow re-ordering in a future release.

Next is the “Formatting” tab. Here’s where you choose either PDF or DocBook. If you choose PDF, you can select a theme, language, etc. Scroll 1.0 has two themes available. In future releases, K15t Software plan to provide more themes and also more options for customisation.

Scroll converts wiki to DocBook and PDF

The Scroll documentation tells you more about the formatting options.

On the “Advanced” tab, you can tell Scroll what to do if it encounters some wiki markup or macros that it doesn’t recognise.

Scroll exports wiki to DocBook and PDF

When you’re ready, you kick off the export. Scroll will show its progress and then present you with a file for downloading.

Scroll exports wiki to DocBook and PDF

If you chose to export as PDF, the output will be a PDF file. If you chose DocBook, it will be a zip file containing the DocBook 5 XML and all images and other attachments.

Some comments from me

I’ve really enjoyed my contact with Tobias and K15t. They’re passionate about their software, passionate about Confluence wiki, and keen to hear feedback. Here are some useful titbits from my experimenting with Scroll:

• Note the {scroll-ignore} macro. You can put it onto your wiki page to tell Scroll not to export a particular chunk of text/content.
• Scroll supports a number of wiki markup elements and macros. The Scroll Authoring Guide tells you how to write your text for best Scroll results.
• Scroll exports DocBook version 5.0.
• The “Scroll Quick Help” tips on the right of the export screens (see above) are useful and concise.
• It’s really useful to be able to determine error handling, telling Scroll what to do when it encounters some wiki markup or a macro that it doesn’t know what to do with. See the “Advanced” tab described above.
• In the PDF export, the “Simple Documentation” theme is neat and attractive.
• For PDF, Scroll 1.0 has two themes available. In future releases, K15t Software plan to provide more themes and also more options for customisation. They’re also planning to allow pluggable themes, so that developers can create their own themes for Scroll.
• I’ve suggested an addition to the Scroll documentation, to provide more information about the DocBook XML produced. The Scroll team have added this to their “to do” list.
• The section on nesting of headings in the Scroll documentation explains why your heading levels may sometimes be different to what you expect.

Some comments from Giles

Giles Gaskell is a technical writer and colleague at Atlassian. He’s had extensive experience with DocBook, so he’s very interested in the Scroll plugin for Confluence too and I’ve been delighted to have his help with the plugin analysis. Here are Giles’s comments:

• I really like the XML indenting that Scroll generates. It makes the XML output very readable via a standard text editor. I’ve worked with several XML editors in the past that force you to use their products to read/edit your XML content, since they either implement their own strange XML indenting or remove all white space > 1 character in length, in their XML output files.
• It’s great that the version of DocBook XML exported by Scroll is 5.0. There are several processing advantages in DocBook XML 5.0 (via the RELAX NG schema) over what’s available in previous versions of DocBook XML (which are based on DTD schema only). Furthermore, it seems that RELAX NG is the way DocBook XML processing is headed.
• I validated both DocBook XML exports using my own installation of jEdit and both were without any errors 🙂
• Scroll does not add a document type declaration at the top of the XML output. It seems that DocBook XML 5.0 files which make use of RELAX NG-based processors typically do not possess a document type declaration. Likewise, it seems that DocBook XML 5.0 output without a document type declaration can only be processed by tools which are RELAX NG-compliant. Hence, it might be useful to have an option in the Scroll Wiki Exporter to add a document type declaration to the XML output, so that XML processing tools which verify XML against DTDs only can process these exports. [Giles would be very interested to hear any comments on this point from other DocBook users.]
• It might also be worth providing an option to choose the version of DocBook XML exported — 4.0, 4.5 or 5.0. However, k15t might consider conducting a brief feasibility study on this first as I suspect integrating this feature would require a fair bit of work and some understanding about the differences between each version of DocBook. The main area of demand for this feature would be customers who have an existing/legacy tool setup that manipulates DocBook XML 4.x content, but do not have resources to upgrade their entire system to DocBook XML 5.0. DocBook XML 5.0 was only officially released in January 2008, so I suspect many DocBook XML customers would still be using DocBook XML 4.x.
• Regarding the use of the <article> root element as opposed to <book> in the DocBook XML output: I believe this is an advantage, as it gives the author more flexibility to structure their books. The <book> element encapsulates information about a book’s content at a higher level than what is contained within an <article>. For example, an author may want to export multiple DocBook <article>s from a Confluence installation. These might be sourced from a variety of Confluence spaces or subtrees. They might then wish to combine these exported DocBook XML <article>s into a single DocBook <book>, using their own DocBook XML editor. If Scroll exported to a DocBook <book>, this would not be possible without additional XML processing.

Giles is also discussing a couple of minor points about the DocBook XML, such as the fact that each <link> element defines its own xlink namespace, and the production of the <?linebreak?> processing instructions.

Try it out yourself

You can either download the plugin for evaluation from the K15t Software site and try it on your own Confluence wiki, or you can use their demo Confluence site to see the pages already there, create your own pages, and then export them to DocBook or PDF. Awesome!

That’s all folks

Both Giles and I enjoy working with the K15t Software guys. They’re very quick to respond to feedback and I’m sure they’d be delighted for more input from other technical writers. 🙂

Community authors and our technical documentation

Shiver me timbers! We’ve decided to throw our official documentation open to updating by outsiders.

This is a big thing, for me and for the other technical writers at Atlassian. It’s also a big thing for many Atlassians and the community developers who work on the code and plugins for the Atlassian applications.

From my point of view as a technical writer and wiki hugger, it’s awesome to be making even greater use of the fact that our documentation is on a wiki.

And from my point of view as a technical writer and perfectionist… Well, “eek” springs to mind 😉

How did it happen?

Our technical documentation is housed and authored on a Confluence wiki. Up til now, in general only Atlassian staff have been granted ‘update’ permissions on the documentation spaces. We’ve been thinking about granting wider update acccess for quite a while. We pride ourselves on being an open company. We all acknowledge that our community developers, partners and other “ecosystem” inhabitants are awesome and can add a lot of value to the documentation. But there was a lot of thinking and organising to do before we could make it happen.

Then at an Atlassian geek gathering, the community developers stated their case fairly strongly. A “community developer” is someone who writes plugins and extensions to our products, but is not an Atlassian staff member. They often have hints and tips that would be useful to other developers. They can post comments to the forums or add comments to the documentation pages, or blog about it on their own blogs. But that’s not the same as having access to the structured, version-controlled, official technical documentation. They may even find something that needs correction in the documentation. No, never! What, never? Hardly ever 😉

Awesome! They recognise the value of our documentation. And they’re right.

So we dove into it with gusto. There were two big things to sort out:

• Intellectual property.
• The impact on the Atlassian technical writers of monitoring the additional updates for correctness etc.

At first, we had an idealistic view: Let’s just throw the documentation open for editing by interested parties, without asking them to sign any agreements. We didn’t want to deter any would-be contributors by forcing them through an unwieldy signup process. But we soon realised that we need to protect both ourselves and the contributors from unforeseen events and possible malicious interference. So we’ve created an Atlassian Contributor License Agreement, based on the Apache Contributor License Agreement.

This was a really interesting exercise. Jason Sprague from Sparke Helmore Lawyers advised us on the wording and ramifications. Thank you for all the help and advice, Jason!

What does this mean for the technical writers?

As one of the Atlassian technical writers, I’m excited, delighted and just a bit daunted by this turn of events.

• Will we be swamped by updates? The technical writers have RSS feeds on all the documentation spaces, so that we can see who has done what to our precious words 😉 Now there will be even more updates to keep track of.
• Thinking about correctness and accuracy, we’ll often have to consult the development team to verify the updates made by community developers.
• And of course, what about consistency of style and spelling? We’ve written a mini style guide. Will it work? Too much, and no-one will read it.
  Too little, and we’ll turn into full-time proof-readers.

While we’re talking about spelling…

Here’s a bit of light relief. Spelling variants are an endless source of interest to technical writers. Debates may get heated or hilarious, and none more so than the tussles between Australian and US spelling proponents. Atlassian is an Ozzie company and our products use Ozzie spelling in their screens, messages, etc. Many of our customers are in the States or other places in the world…

The Atlassian JIRA team recently had a “sad day” when “Australia’s cultural imperialism took another step backwards” and they grudgingly included a US language pack for JIRA. Here’s the blog post.

Igor Minar tells me that technical writers in the US get all “wrinkly” when they see the Ozzie spelling in the Confluence user interface:

The tweets are here: Igor, Sarah, Igor. LOL, thanks Igor!

Getting back to the community authors

The awesome side of opening up our documentation far outweighs the worries:

• We’ll be getting up-to-date technical information direct from the developers, often keyed in as they code. This is especially valuable for the more technical documentation, such as the Atlassian Plugin Framework and soon the Atlassian Gadgets too.
• We’re using the Confluence wiki at its best — updates by many, monitoring via RSS feeds, bottlenecks unplugged.
• I’m hoping that the new contributors will enjoy updating the documentation and feel more ownership of it and of the applications too. Anne Gentle has an interesting related post about User-generated content versus community-generated content. And one of our community authors has already blogged about the opportunity to contribute to the Atlassian documentation: A truly open company. Wow, that was fast 🙂

A trial period first

Just in case the worries get the upper hand over the awesomeness, we have decided to run a trial for three months. Then we’ll take a look at the extra workload, the feedback from the contributors, the quality of the updates, and so on.

I’m so hoping for a successful trial with lots of awesome updates!

CC-by

While we were busy with IP-related things, we sorted out the copyright on our documentation too. We’ve stamped the documentation with a Creative Commons Attribution licence. To the technical writers, this is not such a big change. It’s more like a confirmation of our unofficial policy. But it’s pretty cool to have it in black and white, and with such an attractive CC-by logo too!

Stamina for more?

If you have a yen for the infamous “more details” (that’s a technical writer insider joke, folks 😉 ) take a look at the post on the Atlassian News blog.

If you’re interesting in contributing to the Atlassian documentation, drop us a line.

Update – Playing some more with DITA2Confluence

This is a quick update on my previous blog post about DITA2Confluence. I have just uploaded an extract from the Encyclopaedia Britannica, provided by Project Gutenberg, to my Confluence site. Fun!

All I had to do was download the sample Encyclopaedia Britannica DITA files, create a properties file telling DITA2Confluence where it could find them, and then run the Ant command.

Update - Playing some more with DITA2Confluence

Here’s the table of contents in Confluence:

Update - Playing some more with DITA2Confluence

There were a few errors, but basically it worked.