Integrating NaturalDocs With SyntaxHighlighter (For The Win!)

In working on some SDK-style developer documentation for FirstPoint, it occurred to me that we needed a way to create some all encompasing documentation which covered not only our code base, but also our markup, our JavaScript controls, CSS, and so on. We currently have most of this stuff in Trac wiki pages, which is a great place to put them, but our Trac deployment is going away and being replaced by Jira…or so I’m told.

In light of this, we needed a way to create portable, useful, developer documentation which included a mix of some auto-generated content and hand crafted documents as well (how-to’s and stuff, which would be really terse if placed in code comment). There aren’t really any do-it-all tools, but I stumbled across NaturalDocs which seemed to be the most well rounded tool of the ones I looked into (i.e. JSDocs, YUI Doc, a few others – it started off as a search for a tool for documenting UI conventions, markup, and script usage for our team) because of the fact that it allowed for the inclusion of loose .txt files which would essentially be treated like wiki pages.

You can see an example of the output at the MapQuest API documentation site.

One of the main reasons I liked NaturalDocs is because of the support for code blocks and how easy it is to write them in the loose text files. However, the downside is that the generated output is pretty…boring. Here’s an example:

You can see that it has no intelligence with regards to the language. Doing a little digging around the ‘Net, I found a ticket for a request for support for syntax highlighting. So I ended up rolling my sleeves up and solving this myself. I decided to integrate against the SyntaxHighlighter JavaScript library since I’ve used it previously and I like the output :-).

Here is the end result:

I hadn’t touched Perl in quite some time (since college), so I had to dig around in there for a bit but I was able to integrate it after a few hours of flailing.

The steps required are as follows (these steps assume you use framed mode):

In the file FramedHTML.pm, you will need to add the following lines to the method BuildFile after the call to $self->ClosingBrowserStyles():

Next, to support the new getters, you will need to modify HTMLBase.pm and add the following lines:

You should add more to handle whatever syntaxes you need to handle.

By default, the code blocks are generated as <blockquote><pre></pre></blockquote>. To support SyntaxHighlighter, we’ll need to change this to allow customizing the class name on the <pre> tag. I decided to use the suggested syntax for including this in the markup: (start code <language>). For example: (start code js). The first module that we have to modify to support this is Native.pm. In the method FormatBody, I made the following change to support the extra token:

You can see that I’ve introduced a capturing group to the regular expression to grab the language type (matching SyntaxHighlighter’s language strings). The next step is to modify the generation of the intermediate <code> tag to include a class attribute.

If you stop here, the output generation doesn’t work correctly since this only affects the intermediate output. We need to jump to HTMLBase.pm and modify the method NDMarkupToHTML so that we can generate the proper tag structure. Here are my modifications:

You can see that here, I changed the regular expression used to split the intermediate output into chunks to properly split on the new markup structure. In the if-statement, I changed the eq comparison to a regular expressoin match with a capturing group and inserted that into the output <pre> tag (for the win!).

Now be warned: this is not a complete fix. While this addresses the major issue, generation of the proper output, I did not make changes to copy the image files and JavaScript files required by SyntaxHighlighter (sorry, you’re going to have to do that yourself :-P). The next set of steps are to:

1. Copy the images associated with SyntaxHighlighter to the \output\styles directory (or whereever you like).
2. Copy the scripts required for SyntaxHighlighter to the \output\javascript directory.
3. Modify the paths in the CSS for the icons used with SyntaxHighlighter (in the shCore.css file).

Of course, the themes and CSS files are easy to include since you can specify those at the command line.

That’s all there is to it! Happy documenting! I’ve attached sample files (including the modified source) for SyntaxHighlighter 1.5 and 2.0.

natural-doc-sh2.0.7z (315.01 KB)
natural-doc-sh1.5.7z (281.7 KB)