Blog Archives

Making the include macro’s content appear in the Confluence search results

Confluence wiki provides two macros that you can use to embed content from one page onto another page: {include} and {excerpt-include}. Technical writers find these macros very handy for content reuse. Write the words once, put them somewhere sensible, and then reuse them in a number of different places. Confluence will dynamically copying the content into the page at display time. The trouble is that the Confluence search results pick up the content where it’s originally written, but not where it’s displayed. Or that’s what I thought until now!

Today I learned that you can make the content of the {include} and {excerpt-include} macros appear in the search results for the page where that content is displayed. Magic!

It would be nice if Confluence itself would offer the option to include the macro contents in the search results. If you agree, please comment on or vote for the improvement request: CONF-19054

In the meantime, there’s a workaround. A big thank you to Emily Johnson and David Peterson for finding and sharing this solution, and to Bob Swift for the Cache plugin for Confluence!

The solution in a nutshell

Install the Cache plugin for Confluence. Then wrap your include macros with the {cache} macro, using the “index=true” parameter to make the content of the include macros searchable.

{cache:index=true}
{include:mySPACEKEY:My page name}
{cache}

More about the Cache plugin

Bob Swift develops and maintains the Cache plugin for Confluence. He also provides the support for the plugin.

What does the plugin do? It’s principal function is to allow you to cache part of the page, especially if the content is derived from an external source. This could significantly improve the loading time of the page. You can configure the length of time that Confluence waits before updating the cached content.

For our purposes, one particular option in the Cache plugin is useful: the “index=true” parameter. There are many macros that you can use to display external content on a Confluence page. The {include} and {excerpt-include} macros are two examples. Even though the content appears on a Confluence page, the content will not be included in the Confluence search results, since it is not included in the wiki markup of the page and is therefore not included in the search index. The {cache} macro with parameter “index=true” will ensure that the content is indexed and therefore appears in the search results. See the plugin documentation.

Installing the Cache plugin

The easiest way to install the plugin is using the Confluence plugin manager:

  1. Go to your Confluence Administration Console by clicking “Browse” > “Confluence Admin”.
  2. Click “Plugins” in the left-hand menu.
  3. Click the “Install” tab.
  4. Select “All Available” in the “Plugins to show” dropdown list.
  5. Find the “Cache Plugin” in the list of plugins and click the plugin name. A panel will open up, showing the plugin description.
  6. Click “Install Now”.

If for some reason you can’t use the Confluence plugin manager, then you can download the plugin and install it manually:

  1. Go to the Cache plugin’s page on the Atlassian Plugin Exchange.
  2. Download the plugin JAR file. It has a name something like “cache-plugin-4.1.0.jar”. (The version number will probably change over time.) Save the JAR file on your local computer.
  3. Go to your Confluence Administration Console.
  4. Click “Plugins” in the left-hand menu.
  5. Click the “Install” tab.
  6. Click “Upload Plugin”.
  7. Browse to find the JAR file that you saved, and then click “Upload” to upload it into Confluence.

Let’s try it out

Let’s see what happens when we search for the terms “chocolate” and “cheese”, where those terms are included via the include macros. First we’ll do it without the {cache} macro, then we’ll add the {cache} macro and try the search again.

Example of search results without the Cache macro

First I created some pages.

1. A page called “Favourite food”

This page is in the “CHUNKS” space. The entire content of this page will be embedded into another page later.

Wiki markup:

{cheese}
I like chocolate too.

Displayed content:

I like cheese!

I like chocolate too.

The {cheese} macro is a bit of fun. It’s a real macro, shipped with Confluence. All it does is generate the words “I like cheese!” 🙂

2. A page called “Partial content reuse”:

This page is in the “Demonstration (DS)” space.  The excerpt defined on this page will be embedded into another page later.

Wiki markup:

This page has an excerpt defined on it.
{excerpt}I'm partial to chocolate.{excerpt}

Displayed content:

This page has an excerpt defined on it.

I’m partial to chocolate.

3. A page called “Include food page from another space”:

This page is in the “Demonstration (DS)” space.  This page will include the entire content from the “Favourite food” page (page 1).

Wiki markup:

*Tell me about your favourite foods.*
{include:CHUNKS:Favourite food}

Displayed content:

Tell me about your favourite foods.

I like cheese!

I like chocolate too.

4. A page called “Include excerpt from another page”:

This page is in the “Demonstration (DS)” space.  This page will include just the excerpt defined on another page (page 2).

Wiki markup:

*What do you really really like?*
{excerpt-include:Partial content reuse|nopanel=true}

Displayed content:

What do you really really like?

I’m partial to chocolate.

5. A page called “This page has chocolate in the title”:

I created this page just to test that the search is working.

Searching for “chocolate”:

I’ve restricted the search to the “Demonstration Space”. We have not yet added the {cache} macro to any of our pages. As expected, the Confluence search shows only those pages that include the word “chocolate” in the title, page content, attachments or labels. It does not pick up any pages where the word “chocolate” is displayed as a result of an include macro.

Making the include macro's content appear the Confluence search results

Search showing "chocolate" only when on page

Searching for “cheese”:

Similarly, the search does not pick up the word “cheese” at all.

Making the include macro's content appear the Confluence search results

Search does not pick up the word "cheese" at all

Example of search results after adding the Cache macro

Next I added the {cache} macro to the two pages that contain include macros. Let the magic begin.

Adding the {cache} macro to the page called “Include food page from another space”:

Wiki markup:

*Tell me about your favourite foods.*
{cache:index=true}
{include:CHUNKS:Favourite food}
{cache}

The displayed content of the page remains the same as before.

Adding the {cache} macro to the page called “Include excerpt from another page”:

Wiki markup:

*What do you really really like?*
{cache:index=true}
{excerpt-include:Partial content reuse|nopanel=true}
{cache}

Again, the displayed content of the page is unchanged.

Now let’s try searching for “chocolate”:

The Confluence search results now show the pages where the word “chocolate” is displayed as a result of an include macro.

Making the include macro's content appear the Confluence search results

The Confluence search picks up "chocolate" from the content of the include macros

Searching for “cheese”:

Yaayyy, the search picks up the word “cheese” too.

Making the include macro's content appear the Confluence search results

The Confluence search picks up "cheese" from the {cheese} macro inside an {include} macro

Doubly magic

That’s right! The Confluence search picks up the word “cheese” that was generated by the {cheese} macro embedded inside an {include} macro!

Update on 18 May 2011: I’ve just heard that the use of the {cache} macro means that the inclusion will ignore any page restrictions that you may have added to the included page. I haven’t tested this myself, and it’s unlikely that you’d want to include a page that has restrictions. But there are some applicable use cases, so please be aware of this possibility.

Content reuse on a wiki – a case study

Recently I finished developing the documentation for a new user directory management framework that’s incorporated into two of our products. When designing the documentation, one of the primary goals was efficient and effective content reuse. Since we do all our documentation on a wiki, this was an interesting and rewarding exercise. So I thought you may like to read about it. 🙂

The reusable chunks of documentation are in the User Management (USERMAN) space on our documentation wiki. The chunks are used in the Confluence administration guide and the JIRA administration guide.

What the documentation is about

The documentation describes a software framework for managing user directories. A user directory is a place where you store information about users and groups. Using the software, administrators can connect various types of user directory to their application, and manage the directory connections. A directory can be an internal directory, with data stored on the product database, or an external LDAP directory like Active Directory, or even another application that manages the user information and directory connections.

Why content reuse makes sense for this documentation

The user directory management framework itself consists of a set of back-end libraries and a user interface plugin. The libraries and UI plugin are currently used in two separate products, Confluence and JIRA. The business rules followed by the back-end process and the UI experience are therefore the same in both Confluence and JIRA. In future, other products may join the party.

It makes sense to share the documentation too. This will ensure a consistent experience for our customers across the products, and improve our efficiency when maintaining the documentation.

Introducing the USERMAN space

The User Management (USERMAN) space contains the reusable chunks of documentation. Its home page looks a little different from our standard documentation home pages. A standard home page introduces readers to the documentation and gives them links to the user’s guide, administrator’s guide, and so on.

Instead, the USERMAN home page:

  • Lets readers know that “This documentation is not intended to be read independently of the JIRA and Confluence documentation” and gives them links to those documentation spaces. This is necessary because many people come to our documentation via Google searches. If they land on this home page, we want them to find their way to the relevant documentation quickly.
  • Tells authors where they can find the reusable chunks within the space.
Content reuse on a wiki - a case study

Home page of the USERMAN space

A simple example of content reuse

Let’s start with a simple topic, configuring the LDAP connection pool. The reusable chunk is in the USERMAN space: _LDAP Connection Pool Settings. Here’s a screenshot of the content of that page:

Content reuse on a wiki - a case study

A chunk of reusable content

The above page consists entirely of reference information. There is no orientation at the top, nor any other context apart from the standard message telling readers where to go for the “real” documentation.

The content of the page is reused in both the JIRA documentation (Configuring the LDAP Connection Pool) and the Confluence documentation (Configuring the LDAP Connection Pool). Here’s what it looks like on the JIRA page:

Content reuse on a wiki - a case study

LDAP connection pool in the JIRA documentation

On the JIRA page shown above, there’s:

  • Contextual information at the top of the page.
  • JIRA-specific “how to” instructions.
  • Reused content: The chunk of reference information, dynamically copied in from the USERMAN space when the page loads into the browser.
  • More contextual information, in the form of “related topics”, at the bottom of the page. (Not shown on the screenshot.)

How do you include content from one page into another?

Confluence wiki provides the {include} macro. To use the macro, you edit your page and enter the macro name in curly brackets, specifying the space key and the page title of the page that you want to include into your page.

For example, the above JIRA page contains this macro code:

{include:USERMAN:_LDAP Connection Pool Settings}

When Confluence renders the page, it will replace the macro code with the content of the “_LDAP Connection Pool Settings” page from the “USERMAN” space.

If you like, you can find out more in the documentation for the {include} macro.

The design of the reusable content

Here’s a summary of various design considerations and decisions:

  • Reusable content in a separate space. I decided to house the reusable chunks in their own space, rather than in one of the product documentation spaces. This is because the information is independent of any specific product.
  • Reusable chunks hidden from the reader. Well, as much as possible. The topics in the USERMAN space are not visible in left-hand navigation bar, but people will find them if they search for them. We need to make it clear to readers that the USERMAN space is not intended for reading independently of the product documentation. We also give readers links pointing to the relevant JIRA and Confluence documentation.
  • Inclusions library. Put the topics in an “inclusions library” and start each page name with an underscore. This will alert both readers and authors to the fact that there’s something different about the page, and will help prevent people from changing the page name (which breaks the auto-inclusion). A while ago, I wrote a post about inclusions libraries. One thing to note is that an “inclusions library” is not a feature of the wiki. It’s just a convention we have adopted. The page called “_InclusionsLibrary” is just a page with a funny name. 🙂
  • Spaces as a mechanism for version control. When we release a significant new version of the user management framework (either the back-end libraries or the front-end user interface) we will need to publish a new version of the documentation too. Then, as each product incorporates the new version of the user management framework, we will adjust the product documentation to include the relevant version of the user management documentation. To make this work, we will create a new USERMANxxx space for each version of the documentation. The “xxx” represents the version number.
  • Size of the reusable chunks. How do we decide how big the reusable chunks should be? Well, this is a biggie, so I decided to devote a whole section to it. See the next section, after the screenshot.

Looking at a tree view of the pages in the USERMAN space, you’ll notice that the space home page (marked with the house icon, and at bottom of the screenshot) has no child pages. In a “normal” documentation space, you’d expect most of the pages to be children of the home page. Instead, in USERMAN all the pages are children of the “_InclusionsLibrary” page.

Content reuse on a wiki - a case study

Tree view showing the pages in the USERMAN space

Deciding the size of the reusable chunks

On the one hand, the smaller the chunks the better. Small pieces of content give us much more flexibility. We can use them in different sequential orders and in different contexts. We can add product-specific and contextual information between the chunks. We can leave out bits in one product and include them in another. We are less affected by changes in the user interface, for example when the order of the input fields changes or entire sections move to different screens.

On the other hand, smaller chunks are more difficult to manage. Every time we need to update the page inclusions, such as when we need to move to a new version, there are a large number of {include} macros to update. This reduces the benefit of efficient maintenance. Too much flexibility may lead to confusion in the final output that the reader sees, reducing the benefit of cross-product consistency.

You could go to one extreme and have a reusable chunk per input field or concept (“setting the optimal size for your LDAP connection pool”). Or you could go to the other extreme and define one large reusable chunk per user task (“connecting an LDAP directory”).

After discussion with the product manager and development team, I decided to go for a middle-of-the-road approach.

  • Most of the reusable content is reference material and conceptual material, rather than “how to” steps.
  • We divided the content into logical blocks, corresponding to a logically-related set of input fields or to a concept.

The result is:

  • For the simpler task-based topics, there are two page inclusions: one for the concept and one for the reference material. Example. (To see the wiki markup source of a page, click “Tools” and select “View Wiki Markup” from the dropdown menu.)
  • For the more complex topics, the number of inclusions ranges from five to ten: one or two conceptual chunks,  a number of reference chunks and one or two illustrative chunks. Example 1. Example 2.

What about the procedures and context that are specific to the product?

In the documentation pages for Confluence and JIRA, there are sections that are specific to the product and sections that are common to both products.

These bits are specific to the product, and therefore not reusable:

  • Procedures: The primary user interface is different for each product, and so the path that the user takes (click here, select this, click that) to get to the directory connection screens is specific to the product. Once the user has reached the directory connection screens, the user interface is the same for both products.
  • Context: It’s a good idea to start each page with a mention of the product (Confluence or JIRA) and other orienting information, so that the reader knows what the page is about. After that, it’s safe to dive into the reused content which, of course, is free of contextual information.
  • More context: The path within the documentation itself differs. For example, we need to give people links to related topics and parent pages.

Each documentation page therefore has a section or sections of reused content, with a wrapping of product-specific content.

Diving into a more complex example of content reuse

Let’s take a look at a complex topic, connecting to an LDAP directory.

The topic has a number of sections:

  • A short introduction to the page.
  • An overview of the LDAP connections provided and why you’d use them.
  • A “how to” procedure.
  • Seven sections containing details of specific settings.
  • Diagrams of possible configurations.
  • Related topics

Only three of the above sections contain content specific to the product:

  • The short introduction.
  • The “how to” procedure.
  • The related topics.

All the other sections are drawn from reusable chunks:

  • An overview of the LDAP connections provided and why you’d use them.
  • Seven sections containing details of specific settings.
  • Diagrams of possible configurations.

To see this at work, go to the page (for example, in the Confluence documentation, Connecting to an LDAP Directory), hover over the “Tools” button and select “View Wiki Markup” from the dropdown menu. You’ll see that most of the page consists of headings and {include} macros like this:

h2. Server Settings
{include:USERMAN:_LDAP Server Settings}
h2. Schema Settings
{include:USERMAN:_LDAP Schema Settings}
h2. Permission Settings
{include:USERMAN:_LDAP Permission Settings}

The first {include} macro draws in a reusable chunk from this page in the USERMAN space: _LDAP Server Settings.

The corresponding page in the JIRA documentation (Connecting to an LDAP Directory) looks very similar to the Confluence page.

Reusing Gliffy diagrams

This is something pretty cool that I discovered when creating this set of documentation. You can use the {include} macro to pull in a page that contains a Gliffy diagram. Let’s say you have a Gliffy macro on Page A. Then on Page B, you insert an {include} macro that pulls in Page A. Confluence will faithfully reproduce the diagram on Page B, as well as Page A.

That’s awesome!

Here’s an example. This page in the USERMAN space contains just a Gliffy diagram and its caption: _Diagram Confluence JIRA Crowd.

Here’s a screenshot of the above page:

Content reuse on a wiki - a case study

A reusable Gliffy diagram

Reusing the diagram:

Developing the design

The exercise of designing the reusable content was interesting in a couple of aspects.

  • First, the end result would be a set of reusable chunks, only loosely resembling the “documentation” that most subject matter experts are used to seeing. How could I prototype the design and the content with them?
  • Secondly, the content would be used in two different sets of documentation, for two different products: Confluence and JIRA. Only the Confluence documentation is “mine”, in terms of technical writing responsibilities. Two other technical writers manage the JIRA documentation. I needed to make sure they had input into the design and were happy with the end result.

The usual process of consulting and collaborating with subject matter experts when designing a new documentation suite goes something like this: Draft the table of contents, review it with product managers and developers, apply changes, add detail, and refine it until we’re all happy.

The difference was that there were two tables of contents:

  • The list of topics and their hierarchical structure as they would appear to readers. Let’s call this the compiled documentation
  • The list of reusable chunks that we would plug together to form the above readable material. Let’s call these the reusable chunks.

My strategy was to discuss and review the planned table of contents for the compiled documentation in great detail, making sure that I covered all topics and was consistent in the amount of attention given to each subject. Then I designed the reusable chunks to best satisfy the requirements so discovered. When designing the reusable chunks, I relied more on my technical writing and information design skills.

As soon as we had a basic design, I created the draft content in the reusable chunks and hooked them together to create the compiled documentation for one of the products.

Then I walked the technical writing team, product managers and developers through that first draft. I showed them the reusable chunks and discussed the design strategy (size of the chunks, use of wiki spaces, setting of permissions, consistency, removal of context, and so on). But most of our attention focused on the human-readable documentation. The review session, as always, gave me much valuable feedback that I applied to the draft.

After that initial review, I focused on each individual topic and went through the usual iterative procedure of drafting, reviewing and updating the content.

The tools

A summary of the tools I’m using for this case study:

  • Confluence wiki.
  • The {include} macro, supplied as part of Confluence.
  • Spaces, the primary way of organising content into logical collections, or libraries, in Confluence.
  • The Documentation theme, supplied as part of Confluence. In fact, you do not need the Documentation theme for content reuse. All of the techniques mentioned in this post will work in the default Confluence theme and most other themes too. But I’ve mentioned the Documentation theme here because the screenshots show it in action, in particular the left-hand table of contents supplied by the theme.
  • Gliffy online diagramming tool. Again, this tool is not needed for content reuse. I’m including it in the list because it’s an interesting part of the case study.

That’s it for now

I hope you find the examples and design tips useful!

More about content re-use on a wiki

A while ago I wrote a post about content re-use on a wiki. Just recently I’ve discovered that the {excerpt} macro has a handy parameter: “hidden=true”. So I thought I’d tell you about it.

In  Confluence wiki, you can dynamically include content from one page into another page. These are the macros to use:

  • Use the {include} macro to include a whole page into another page.
  • Use the {excerpt} macro to define a re-usable chunk of text (an ‘excerpt’) that is part of a page, then include that excerpted text into another page using the {excerpt-include} macro.

My recent “eureka” moment happened when I noticed the “hidden” parameter in the {excerpt} macro:

{excerpt:hidden=true}Blah blah blah.{excerpt}

If you specify “hidden=true”, the content between the {excerpt} tags will not be displayed on the page where the excerpt is defined. It will, however, be displayed on the page where the excerpt is used via the {excerpt-include} macro. Magic!

Here’s an example of a page that does just that. It’s from our SharePoint Connector documentation: _Access Confluence using IWA with IIS

I’ve added some text to the page, explaining that the content is hidden and telling people how they can see it if they really want to. Here’s a screenshot of the same page (click the image to expand it):

More about content re-use on a wiki

More about content re-use on a wiki

Why on earth would you want to hide the content of an excerpt?

It stems from the way we are using content re-use in our structured technical documentation. My earlier post goes into it in detail. Here’s a quick summary.

We create a special “area” of the wiki, which we call the “inclusions library”. It’s not really anything special. It’s just a set of pages that hold the chunks of content (text, images, etc) that we will use more than once, on various wiki pages. We also use an underscore at the start of the page names, to remind us that the pages are special. This helps to prevent us and other authors from editing the included content by mistake.

The thing about a wiki, though, is that people may stumble across the pages in our “inclusions library”. They may wander in via Google search, and not realise that this page is just part of a set of instructions. Sure, it’s got a weird page name. But hey, it’s a wiki. What do you expect?

It may even be a bit traumatic for them if they start following the instructions, without having performed the preceding steps. 😉

Here’s one of the pages that use the excerpt from the above page. This page is neatly in place in the full installation sequence: Access Confluence using Integrated Windows Authentication via IIS with SP 2010. Much less chance of our poor Googler going off the beaten track.

I think the “hidden=true” parameter is pretty neat. I don’t know why I didn’t notice it before. Now you know about it too!

%d bloggers like this: