<CharlieDigital /> Programming, Politics, and uhh…pineapples

8Aug/11Off

TFS – Does It Suck?

Posted by Charles Chen

I'm not sure, but I don't want to find out either.

I'm currently tasked with recommending a new source control platform and a new defect tracking platform as well.

I'm late to this post from March 2010, but Martin Fowler posted an internal ThoughtWorks survey of version control tools:

I conducted the survey from February 23 2010 until March 3 2010 on the ThoughtWorks software development mailing list. I got 99 replies. In the survey I asked everyone to rate a number of version control tools...

...there's a clear cluster around Subversion, git, and Mercurial with high approval and a large amount of responses. It's also clear that there's a big divide in approval between those three, together with Bazaar and Perforce, versus the rest.

The biggest offender?  TFS with a 0% (yes, z-e-r-o) approval from the ThoughtWorks staff.  Scary.

James McKay provides an interesting take on it:

Team Foundation Server advocates claim it’s unfair to compare TFS to other source control tools, since it’s not just source control, but an integrated end-to-end application lifecycle management solution. Comparing TFS to, say, Subversion, is like comparing Microsoft Office to Notepad, so they say.

Now where have I heard something like that before? Oh yes, Lotus Notes:

The main focus for frustration is Notes’s odd way with email, and its unintuitive interface. But to complain about that is to miss the point, says Ben Rose, founder and leader of the UK Notes User Group (www.lnug.org.uk). He’s a Notes administrator, for “a large automotive group”.

It’s regarded by many as an email program, but it’s actually groupware,” Rose explains. “It does do email, and calendaring, but can host discussion forums, and the collaboration can extend to long-distance reporting. It will integrate at the back end with huge systems. It’s extremely powerful.”

The thing is, it wasn’t the detractors who were missing the point. It was the Lotus Notes guys. You see, e-mail is right at the heart of any groupware application. It’s the part of the application that users interact with the most. It’s where usability matters the most. And it’s what Notes got wrong the most.

Is TFS really that bad?  I haven't used it or recommended it (mostly out of concern for cost), but 0% approval?

On a related note, I've been digging into Redmine the last few days to try to examine its suitability for a project that I'm taking over and new products that I'll be bringing online.  I've been really impressed with it, even compared to the excellent Trac.  Compared to Trac, Redmine just feels more well put thought out (i.e. native support for multiple types of source control systems, native sub-projects, so on) and the UI is a bit cleaner and easier to use.  I expect to be blogging about it frequently in the coming months.

Filed under: DevTools, Random Thoughts, Technology 4 Comments
13Jun/11Off

FFMPEG and Flowplayer

Posted by Charles Chen

Flowplayer is a great little bit of Flash and JavaScript for incorporating streaming video on your site (until we get a fully viable, widely supported HTML5 solution).

I've used it in various capacities for putting together apps to stream recorded webcasts using CamStudio and splicing video and audio clips together in Windows Live Movie Maker (plenty adequate for my simple usage).  You can see an example of this on the home page for zaanglabs.

The input video was a 720p .wmv file output from Movie Maker that comes in at 13.7MB.  The output is a 3MB h.264 encoded .flv that is almost as good as the original.

For anyone doing webcasts using open source tools (or you just need to encode whatever you output from Movie Maker), this is what you'll need to pull it all together:

FFMPEG -i input.wmv -ar 44100 -qscale 9 -vcodec libx264 output.flv

The input is the file saved from Movie Maker and the output is a file that can be streamed using Flowplayer.  That seems to be the secret sauce after playing around with various settings and different configuration options.  This yields an output video that should be nearly indistinguishable from the original for webcasting purposes.  I'm sure you'll have to adjust the value if you have more action in your frames.

Happy webcasting!

Filed under: DevTools, Random Thoughts No Comments
1Oct/10Off

Irony .NET Language Implementation Kit

Posted by Charles Chen

I came across Irony (http://irony.codeplex.com/) today while contemplating whether to use antlr or not for a project I'm working on where the requirements call for allowing users to write small conditional instructions.

From the project site description:

Irony is a development kit for implementing languages on .NET platform. It uses the flexibility and power of c# language and .NET Framework 3.5 to implement a completely new and streamlined technology of compiler construction.

Unlike most existing yacc/lex-style solutions Irony does not employ any scanner or parser code generation from grammar specifications written in a specialized meta-language. In Irony the target language grammar is coded directly in c# using operator overloading to express grammar constructs. Irony's scanner and parser modules use the grammar encoded as c# class to control the parsing process. See the expression grammar sample for an example of grammar definition in c# class, and using it in a working parser.

Compared to antlr, it seemed much simpler from the samples.

In the past, I've usually used Spring.NET's expression evaluation functionality (built on antlr); however, the entirety of the Sprint.NET library seemed too heavy for the simple scenario I had to implement and I'd still have to do some string parsing anyways if I used it.  So I set out to try out Irony for myself instead.

The basic gist of the solution is that the interface needs to allow users to specify meta-instructions as strings in the form of:

if ("property1"="value1") action("param1","param2")

Once the user has configured the instructions for a given template document, the meta-instructions are executed when an instance of the template is created and metadata properties are set on the document (in SharePoint).  So the goal is to define a set of meta-instructions and actions which allow users to build dynamic document templates.  For example:

if ("status"="draft") delete()
if ("status"="published") lock()
if ("status"="pending") insert("22ad25d6-3bbd-45f3-bc63-e0e1b931e247")

The first step is to define the grammar (I'm sure this isn't very well constructed BNF, but I need to brush up on that :-D ):

using Irony.Parsing;

namespace IronySample
{
    public class AssemblyDirectiveGrammar : Grammar
    {
        public AssemblyDirectiveGrammar() : base(false)
        {
            // Terminals
            StringLiteral property = new StringLiteral("property", "\"");
            StringLiteral value = new StringLiteral("value", "\"");
            StringLiteral param = new StringLiteral("param", "\"");
            IdentifierTerminal action = new IdentifierTerminal("action");

            // Non-terminals
            NonTerminal command = new NonTerminal("command");
            NonTerminal ifStatement = new NonTerminal("ifStatement");
            NonTerminal comparisonStatement = new NonTerminal("comparisonStatement");
            NonTerminal actionStatement = new NonTerminal("actionStatement");
            NonTerminal argumentsStatement = new NonTerminal("argumentsStatement");
            NonTerminal parametersStatement = new NonTerminal("paremeters");
            NonTerminal parameterStatement = new NonTerminal("parameter");

            // BNF
            command.Rule = ifStatement + NewLine;
            ifStatement.Rule = ToTerm("if") + comparisonStatement + actionStatement;
            comparisonStatement.Rule = "(" + property + "=" + value + ")";
            actionStatement.Rule = action + argumentsStatement;
            argumentsStatement.Rule = "(" + parametersStatement + ")";
            parametersStatement.Rule = MakePlusRule(parametersStatement, ToTerm(","),
                        parameterStatement) | Empty;
            parameterStatement.Rule = param;

            MarkPunctuation("if","(", ")", ",", "=");

            LanguageFlags = LanguageFlags.NewLineBeforeEOF;

            Root = command;
        }
    }
}

This defines the elements of the "language" (see the Irony wikibook for a better explanation).  (You'll note that the "if" is entirely superfluous; I decided to leave it in there just so that it would make more sense to the expression authors as they create the meta-instruction.)

As you're writing your grammar, it'll be useful to test the grammar using the provided grammar explorer tool to check for errors:

Irony grammar explorer tool

Once the grammar is complete, the next step is to make use of it.

I wrote a simple console program that mocks up some data input:

private static void Main(string[] args)
{
    // Mock up the input.
    Dictionary<string, string> inputs = new Dictionary<string, string>
                                        {
                                          {"status", "ready"}
                                        };

    // Mock up the instructions.
    string instructions = "if (\"status\"=\"ready\") Echo(\"Hello, World!\")";

    Program program = new Program();
    program.Run(inputs, instructions);
}

The idea is to simulate a scenario where the metadata on a document stored in SharePoint is mapped to a dictionary which is then passed to a processor.  The processor will iterate through the instructions embedded in the document and perform actions.

In this example, I've mapped the statement directly to a method on the Program class for simplicity.  The action is a method called "Echo" which will be fed on parameter: "Hello, World".  The Run() method contains most of the logic:

private void Run(Dictionary<string, string> inputs, string instructions)
{
    // Run the parser
    AssemblyDirectiveGrammar grammar = new AssemblyDirectiveGrammar();

    LanguageData language = new LanguageData(grammar);

    Parser parser = new Parser(language);

    ParseTree tree = parser.Parse(instructions);

    List<ParseTreeNode> nodes = new List<ParseTreeNode>();

    // Flatten the nodes for easier processing with LINQ
    Flatten(tree.Root, nodes);

    var property = nodes.Where(n => n.Term.Name == "property").FirstOrDefault().Token.Value.ToString();
    var value = nodes.Where(n => n.Term.Name == "value").FirstOrDefault().Token.Value.ToString();
    var action = nodes.Where(n => n.Term.Name == "action").FirstOrDefault().Token.Value.ToString();
    string[] parameters = (from n in nodes
                           where n.Term.Name == "param"
                           select Convert.ToString(n.Token.Value)).ToArray();

    // Execute logic
    string inputValue = inputs[property];

    if(inputValue != value)
    {
        return;
    }

    MethodInfo method = GetType().GetMethod(action);

    if(method == null)
    {
        return;
    }

    method.Invoke(this, parameters);
}

You can see that in this case, the evaluation is very simple; it's a basic string equality comparison.  The action execution is basic as well.  It simply executes a method of the same name on the current object instance.  Try running the code and changing the "ready" value in the dictionary and see what happens.

A helper method is included to flatten the resultant abstract syntax tree for querying with LINQ (could possibly be done with a recursive LINQ query?):

public void Flatten(ParseTreeNode node, List<ParseTreeNode> nodes)
{
    nodes.Add(node);

    foreach (ParseTreeNode child in node.ChildNodes)
    {
        Flatten(child, nodes);
    }
}

And finally, the actual method that gets invoked (the action):

public void Echo(string message)
{
    System.Console.Out.WriteLine("Echoed: {0}", message);
}

This sample is fairly basic, but it was pretty easy to get up and running (far easier than antlr) and there's lots of potential for other use cases.  One thing I've found lacking so far is documentation.  It's fairly sparse so there's going to be a lot of trial and error, but the good news is that the source code includes a lot of examples (some of them fairly complex including C#, SQL, and Scheme grammars).

Filed under: .Net, DevTools No Comments
5Apr/10Off

jsTree and Nested XML Data Stores

Posted by Charles Chen

I happened upon jsTree a few months back while searching for a solid jQuery based tree.

Without a doubt, it is one of the most well implemented and functional Javascript trees I've used with perhaps the most powerful feature being the built-in support for client-side XML representations of the tree and the ability to add arbitrary metadata to the tree using the Sarissa library.

While the tree itself is extremely powerful, some of the documentation is actually out of date and made my implementation of the tree a bit more tasking than it should have been.

For example, the option for initializing the tree with a static XML string (as demonstrated here) actually requires using staticData instead of static in the latest version.  Adding metadata to the tree is also made far more complicated by the documentation and examples I found online.

In reality, the datastore implementation for the nested XML support is powerful enough to extract arbitrary DOM attributes (online posts seem to indicate that you need to use the custom metadata plugin and/or the jQuery metadata plugin).  You can see in the sample below, that I set the attribute "md" to an encoded JSON string (you'll want to do this for when you reload the tree as a Javascript string) and then retrieve the value as a part of the XML by specifying the attributes to collect:

/*--- test adding data ---*/
$("#test").click(function() {
    var t = $.tree.focused();

    if (!t.selected) {
        return;
    }

    /*--- sets a person on the node ---*/
    var person = {
        "FirstName": $("#firstName").val(),
        "LastName": $("#lastName").val(),
        "Age": $("#age").val() 
    };

    var serialized = JSON.stringify(person);

    /*--- sample of adding an arbitrary attribute at the DOM level ---*/
    t.selected.attr("md", encodeURI(serialized));

    /*--- ...and how to retrieve it in XML ---*/
    var opts = {};
    opts.outer_attrib = ["id", "rel", "class", "md"];

    var xml = t.get(null, "xml_nested", opts)

    $("#xml-d").text(xml);
    $("#treeXml").val(encodeURI(xml));              
});

In real usage, you'd assign some more meaningful values to the metadata and save it.  I find that with a library like this, it's probably easier to just save the whole XML string and that's simple enough by just pushing the XML to a hidden input before submitting the form (as I've done in the last line).

Mahr Mohyuddin has a much more complex and more generic implementation of ASP.NET integration here, but I think that might be more complexity than is needed.  In practice, it makes more sense to use full JSON objects on the client side (as I've used above) and embed them into the attribute and then, using the JavaScriptSerializer class, extract the objects into domain objects on the server side.  Here's an example:

string xmlString = Uri.UnescapeDataString(treeXml.Value);

XDocument xml = XDocument.Parse(xmlString);

// Get all the <items/>.
var items = from x in xml.Descendants()
            where x.Name == "item"
            select x;

List<Person> people = new List<Person>();

JavaScriptSerializer serializer = new JavaScriptSerializer();

// Resolve the paths and Person instances.
foreach(var item in items) {
    string[] parts = item.AncestorsAndSelf()
        .Select(a => a.Descendants("name").First().Value)
        .Reverse().Skip(1).ToArray();

    string path = string.Join("/", parts);

    if(item.Attribute("md") == null) {
        continue; // Next iteration.
    }

    string serializedPerson = 
        Uri.UnescapeDataString(item.Attribute("md").Value);

    Person p = serializer.Deserialize<Person>(serializedPerson);
    p.OrgPath = path;

    people.Add(p);
}

_people.DataSource = people;
_people.DataBind();

Nothing fancy here; the only thing of note is the little LINQ query to resolve the node path (may or may not be useful).

The Person class is also very simple and barebones:

using System;

namespace JsTreeSample {
    /// <summary>
    /// Models a person.
    /// </summary>
    /// <remarks>
    /// Serializable to support deserialization from JSON.
    /// </remarks>
    [Serializable]
    public class Person {
        private string _orgPath;
        private int _age;
        private string _firstName;
        private string _lastName;

        public string FirstName {
            get { return _firstName; }
            set { _firstName = value; }
        }

        public string LastName {
            get { return _lastName; }
            set { _lastName = value; }
        }

        public int Age {
            get { return _age; }
            set { _age = value; }
        }

        public string OrgPath {
            get { return _orgPath; }
            set { _orgPath = value; }
        }
    }
}

The full project is included.  Some usage notes: select a node first and then enter a first name, last name, and age.  Then click "Set Data" to create a Person object at the node (this will also show the full XML of the tree).  Then click Submit to send the data (displays in a repeater).

JsTreeSample.7z (144.07 KB)

Definitely check out jsTree for your next project; it's amazingingly versatile and rich in functionality.

Filed under: Dev, DevTools 1 Comment
2Apr/10Off

SharpZipLib and ASP.NET

Posted by Charles Chen

I recently had to write a search-driven component to extract and export documents from a SharePoint repository. It presented a challenge since many examples on the web start from the premise of a file system and not binary streams.

I settled upon SharpZipLib, an excellent and fairly easy to use library, but found the documentation quite lacking, particularly around creating a zip file to a stream (like an ASP.NET output response stream).

I put together a little sample just to test it out and finally got it working after struggling with it for a good 30 minutes. This sample creates a zip package using an embedded resource (text file) for simplicity. In practice, you can just modify the implementation of the GetBinaryContent helper method. Note that when adding multiple files, you do a "put" first and then a "write".

using System;
using System.IO;
using System.Reflection;
using System.Web;
using System.Web.UI;
using ICSharpCode.SharpZipLib.Zip;

namespace SharpZipLibTest {
    public partial class _Default : Page {
        protected void Page_Load(object sender, EventArgs e) {
            string fileName = "package.zip";

            // Clear the response.
            Response.Clear();
            Response.Buffer = true;
            Response.ContentType = "application/octet-stream";
            Response.Charset = "";
            Response.AddHeader("Content-Disposition",
                string.Format("attachment; filename={0}", fileName));

            using (var ms = new MemoryStream()) 
            using (var zip = new ZipOutputStream(ms)) {
                byte[] fileBuffer = GetBinaryContent();

                // Write to the zip package.
                var entry = new ZipEntry("helloworld.txt");
                entry.DateTime = DateTime.Now;
                entry.Size = fileBuffer.Length;
                zip.PutNextEntry(entry);
                zip.Write(fileBuffer, 0, fileBuffer.Length);

                // Repeat for each file
                /*
                for(...) {
                    var entry = new ZipEntry("...");

                    zip.PutNextEntry(entry);
                    zip.Write(...);
                }
                */

                zip.Flush();
                zip.Finish();

                byte[] output = ms.ToArray();

                // Write the final output to the response stream.
                Response.OutputStream.Write(output, 0, output.Length);                
            }

            // End the response.
            Response.Flush();
            Response.End();

            HttpContext.Current.ApplicationInstance.CompleteRequest();
        }

        /// <summary>
        /// Gets the binary content to send.
        /// </summary>
        /// <returns>The byte array containing the binary content.</returns>
        private byte[] GetBinaryContent() {
            string resourceName = "SharpZipLibTest.helloworld.txt";

            byte[] fileBuffer;

            using (Stream s = Assembly.GetExecutingAssembly()
                .GetManifestResourceStream(resourceName)) {
                // Read the file stream into a buffer.
                fileBuffer = new byte[s.Length];

                s.Read(fileBuffer, 0, fileBuffer.Length);
            }

            return fileBuffer;
        }
    }
}

The full project file is attached and runnable.

SharpZipLibTest.7z (104.3 KB)

Filed under: .Net, DevTools No Comments
26Feb/10Off

SharePoint 2010 Mobile Rendering (And Device Emulator Fun!)

Posted by Charles Chen

I was recently tasked with taking an exploratory look at the mobile rendering capabilities of SharePoint 2010 to get a feel for it.  The short of it is that it's sad to say that in 2010 (the year, that is), we are still doing not much better than 3-4 years ago.  Of course, this raises the question of whether it's an issue with the browser or an issue with the software (SharePoint 2010).  Based on my early experience, I have to say it's an issue with 2010 - it seems like the SharePoint team simply took the easy way out.

If you've ever used the Gmail web page from a mobile browser (like Opera), you'll see that there is a "mobile" version and a "basic HTML" version available (links on the bottom).  For all intents and purposes, the "basic HTML" version looks and works pretty much like the full web version (minus the fancy drag/drop stuff, chat, etc.) and loads fine on my nearly two year old HTC Touch Pro.  This is what we should expect from a company as big as Microsoft with vast resources at their disposal.  This is what we should expect from them given that it's 2010 and the mobile space is an increasingly bigger piece of the pie. 

What's worse is that there's no apparent way from the browser client to access the "full view".  What happens when more capable mobile browsers are deployed that can match the capabilities of the current desktop browsers?  Seems like you'll have to make configuration changes on the server to enable proper browser detection but it would be much easier to simply have the option, like on mobile Amazon.com, to load the full view in lieu of Microsoft actually offering a more usable "basic HTML" view.

Alas, instead, we get neutered functionality and weak rendering to mobile browsers for a product that's supposed to carry us to 2013 or 2014.

The following screenshots are taken using my desktop IE8, Microsoft's Windows Mobile 6.1.4 emulator, and RIM's Blackberry 9630 emulator:

Landing Page

List View

Calendar View

New Event

In analysis, I think what makes matters worse is that they've dramatically increased the capabilities of the desktop browser version, but neglected to carry forward some of the basic functionality that seems like it wouldn't be too had to implement like the ratings or tagging functionality.  This discrepancy makes the shortcomings of the mobile offering all the more apparent.

Certainly, I'm not expecting that the desktop experience to be fully emulated on a mobile device, knowing the limitations of the small screens and limited resources and capabilities of mobile browsers.  That said, I would expect a more worthy effort than this.  I would expect something similar to Gmail's "basic HTML" option and a link to the full version.  It's an embarassment, especially the new event screen; at the least, the developers could have aligned the fields (it's pretty offensive).

It's just sloppy and lazy all around and I suggest Microsoft spend some time with the mobile browsers from their competitors.  People are accustomed to a certain level of mobile browser capabilities these days and it's nowhere near as low as Microsoft would like to believe.

For comparison's sake, here's Gmail's rendering in "mobile" and "basic HTML" mode:

Gmail desktop browser, "mobile", and "basic HTML" rendering

This is much more usable and a much more congruous experience across platforms.  Come on Microsoft, stop being lazy!  As Mark Jackson would say "You're better than that!"

On a side note, working with the emulators was a bit challenging.  If you'd like to try it out for yourself, here are some resources:

Mobile IE 6

Blackberry Devices

My experience is that working with the Blackberry emulator is extremely frustrating because you'll want to click on the screen with your mouse but then you remember that the devices don't support touch so it would only be logical that the emulator doesn't either.  However, this logic leads to a hair-pulling experience when working with the emulator.

Filed under: DevTools, Rants, SharePoint 1 Comment
4Dec/09Off

More WebSequenceDiagrams.com Awesomeness

Posted by Charles Chen

As I've been working with a client which has demanded rigorous sequence diagrams as deliverables for the design phase of the project, I've started to use WebSequenceDiagrams.com more and more.

I've blogged about it previously, but I've only come to truly appreciate it after having to use Visio for a few days before I convinced the client that I could deliver the content faster and in an easier to maintain format (well, text) using WSD.

What Visio Gets Wrong

I ran out of connection points. Yes, it's possible; for a long activation sequence, you can actually run out of connection points. One could argue that this calls for a refactoring of the diagram in the first place, but then I would say that you've never tried to actually refactor a Visio diagram...I still find it hard to believe: the activation box ran out of connection points; I wouldn't have believed it if I didn't see it myself:

I kept having to resize the page. By default, you can't really fit much on the page. But as I started to build my sequence up, I found that I had to keep toggling around with the paper size just so that I would have a grid. This was annoying since it also involved then zooming out so that I could select everything and reposition it to the top left corner of the page then zooming back in. You'd think that Visio would be smart enough to do this, but it isn't...

I kept having to move elements around. Want to add a new step? What about introducing a new actor in between two existing actors? Prepare for some carpal tunnel my friend. There's simply no easy way to do it aside from zooming out, grabbing everything to the right and shifting it around while counting gridlines and getting your result some 10-20 clicks later. What's worse is that you end up having to scroll around horizontally (reordering actors) and vertically (reordering steps) while dragging a bunch of stuff around.

I kept having to fix connection lines. This was absolutely mindboggling: if I extended an activation, it would cause the first connection on the activation to jump, which would then require me to manually drag the connection back to where it belonged. I probably spent a good 10% of my time simply fixing these connection points as a adjusted activations:

This is an incomprehensible design flaw; I have no idea how people work around this in Visio since I adjust activations multiple times as I'm working through a diagram.

There was no representation of alternate paths or optional steps. I ended up having to draw a rectangle and manually managing the size of it as I changed steps and added more steps. What made this even more annoying was that having the rectangle, even though I sent it to the back, then made it difficult to select elements that were enclosed by the rectangle like a message line or a connection point or a note; I'd end up selecting this stupid rectangle instead.

I had to keep managing the location of notes. The notes don't seem to anchor to anything and it's not clear to me how that's supposed to work. That meant that I had to keep moving notes around as I changed activations and modified connection points.

The lifelines didn't synchronize. For the life of me, I couldn't figure out how to get the lifelines to synchronize in length so I didn't have to manually go back and drag each one down to the same length. You can't actually CTRL+click two lifelines, as one would think you'd be able to do, and drag them both to extend them simultaneously. I mean, this seamed like pretty basic stuff to me.

So yeah, all in all, I'm not sure why anyone would want to subject themselves to the pain of creating sequence diagrams in Visio. Maybe if it's a final product and you don't plan on touching it ever again and you've already done most of the work on paper or something, but it's a terrible tool if you're just trying to think an idea out and see it visually.

What WebSequenceDiagrams.com Gets Right

I can't speak to the console program or the DLL (yet), but I decided that the only way that I could do this right was to do it in WSD first and then just use Visio to render the final output. While doing it in the browser is fine, I found it much easier to do it in EditPlus, my text editor of choice.

The first step was to create a syntax and auto-complete file so that it was a little more user-friendly.

Here's my .stx syntax file:

#TITLE=WSD

#DELIMITER=,(){}[]+*%/="'~!&|<?:;.
#QUOTATION1="
#LINECOMMENT=note
#CASE=y

#KEYWORD=Activate
activate

#KEYWORD=Deactivate
deactivate
destroy

#KEYWORD=Alias
as
participant

#KEYWORD=Notes
note
over
right
left

#KEYWORD=Control
alt
else
end
opt

#KEYWORD=Newline
\n

#KEYWORD=Transition
->
-->
#

And my .acp auto-complete file:

#TITLE=WSD

#CASE=y

#T=act
activate ^!
#T=de
deactivate ^!
#T=pa
participant ^!
#T=alt
alt ^!

else

end
#T=opt
opt ^!

end
#

What I get from this is:

I know what you're thinking: Chuck, that looks like code! By golly, it does! And -- at least to me -- that's the beauty; all of a sudden, a frustrating, time consuming, mouse-centric activity becomes a keyboard-centric, coding-like activity. Moving objects around becomes a matter of moving lines of text. Reordering actors involves moving your participant declarations around. Notes stay in their context if you add a step since everything gets pushed down. There's no manual resizing of anything. There's no fixing connection points. There's no stupid. It actually makes working with sequence diagrams, beyond just whiteboarding, much more useful and much more productive as it lets you kind of think out the code by actually writing pseudocode.

I highly recommend downloading EditPlus (you can keep using it for free, perpetually, if you're a cheap bastard or pony up the $20 for such an awesome editor). For me, EditPlus is a perfect pairing for WSD due to the easy to create language syntax/autocomplete files and the handy split-document feature so you can easily reference your participants at the top.

Simply create a new file type and add the .stx and .acp files I defined:

Now the downside of this whole affair is that you have an extra step of having to copy out the text and pasting it into the browser, but even with that extra little bit of annoyance, the time and frustration saved over working with Visio is more than worth it.

The next step, once I get my hands on the command line tool, is to hook it up to the external tools feature of EditPlus for a quick hit. I'm also considering writing an integration for Visual Studio for custom rendering within VS or at least something quick-and-dirty like an add-in.

One additional note is that WSD has an HTML/Javascript API whereby you can render a diagram inline with your HTML by simply using a set of <pre></pre> tags and a reference to a Javascript file.

What's cool about this is that now you can use the standard CTRL+B/CTRL+E shortcut keys to preview without a copy/paste step! For free! That's pretty awesome.

Of course, the downside is that using this method, there is a limit to the size of the sequence that you can send up as well as the fact that you need some additional hijinks to make the syntax highlighting work (I gave up on that part :-D ) . But if were doing it in the browser to begin with or using notepad and you don't care for the syntax highlighting, then this is a huge upgrade.

Conclusion: stop using Visio :-D Now I'm just looking forward to WebERDiagrams.com, WebStateMachineDiagram.com, and....well, you get the idea.

My EditPlus .stx and .acp files for anyone that wants 'em: wsd-files.zip (.52 KB)

Filed under: Awesome, DevTools No Comments
22Jul/09Off

Mercurial vs. SVN

Posted by Charles Chen

As I've been starting on a new project recently, I've been delving into Mercurial (hg) as an alternative to Subversion (svn).

I've been using svn for about 3 years now, and - for the most part - it has been way better than Visual Source Safe (whatever last version I used...it sucked), it clearly has some pain points that make daily workflows difficult.  The most important of these are the speed of commits, the fact that a commit is global, reverting to a version, and branching/merging (painful...so painful).

hg addresses each of these pain points:

  • Speed of commits and local commits: Mercurial has two separate concepts of commit and push.  A commit in hg creates a local commit as opposed to a global commit in svn.  This is useful because you can work->work->commit->work->commit->work->revert->commit without affecting anyone else's work stream.  In other words, you don't have to wait to commit until you have flawless, working, compiling code.  You can commit as much as you want and keep your work intact; this is a big win as it encourages experimentation.  When you're ready to share your code, a push operation (the equivalent of a svn commit) pushes it to a shared location for others to pull.  Of course, to some degree, you can accomplish this with svn as well using a branch, but...
  • Branching/merging: OMG, so much easier and more intuitive than svn.  I can't believe it.  In comparison, svn is a giant charlie foxtrot.  Branching and merging in svn pre-1.5 was an exercise in futility.  It was extremely difficult to remember the right procedure (always requiring a lookup to the docs) and very much error prone.  So difficult, in fact, that it discouraged branching for fear of wasting a good half day trying to merge it back in later.  Maybe it's better with 1.5?
  • Reverting to a version: ever try it in svn?  'Nuff said.  It's counterintuitive and always confuses the heck out of junior devs or devs new to svn.

I've only been using it for a few days now (primarily TortoiseHg), so perhaps hg has just as many warts as svn, but I'm going to stick with it and find out.

Some good resources on hg/git vs. svn (or DCVS vs CCVS):

I'll post new entries as I learn more about hg in daily use :)

P.S. WebFaction is a pretty awesome webhost.  For the low, low price of some $10 a month, I can host my own hg repository, svn repository, Trac, and (note: not or) my web app.  Hot damn, that's awesome!  Although svn was significantly easier to set up with WebFaction than hg, I've read that they are close to officially supporting hg and it should be just as easy in the future.

Filed under: Dev, DevTools No Comments
27May/09Off

Random DevTools Entry #017

Posted by Charles Chen

In software development, it's incredibly useful to be able to visualize your code interactions using sequence diagrams and data flow diagrams and what not.  Not only is the visualization a plus, the act of generating one helps tremendously in terms of working out the outline of the logic that you need to implement.  One of the biggest problems I've come across in this area are the tools: they're simply too heavy and too complex for general diagram drawing tasks in addition to being generally rigid as well.


Today, as I was about to download and install Visio, I decided to spend a few minutes checking to see if there was a web alternative.  Enter websequencediagrams.  This is an all around awesome little tool to add to your toolbox.  Not only is it free (free is always awesome), it's text based.  At first blush, this seems terrible; there's a whole syntax to learn and lots of typing.  But the syntax is incredibly easy and simple while powerful and easy to understand.


Here's an example (I've bolded the syntax to make it easier to distinguish):



Browser->App: HandleSearchClickEvent()
App->Service: ExecuteSearch(keyword)
activate Service
Service-->App: (return results)
deactivate Service
App->App: RenderResults(reseults)
note right of App:
    render URL with
    keyword in query string.
end note
App-->Browser: HTML
Browser-->Office: click: http://../name.docx?term=keyword
Office->AddIn: ThisAddIn_Startup()
AddIn->AddIn: Check for search term
note left of AddIn:
    ActiveDocument.FullName will
    contain the query string.  This
    can be extracted with a regular
    expression
end note
AddIn->AddIn: Execute find


And here is the result:



There are also a variety of pre-defined styles you can choose to render your diagram.  It's all sorts of awesome and a real time-saver compared to traditional tools.  I personally love that it's text based; I've found that when working with Visio (and other such tools), more than half of my time is spent arranging things just to get them to line up.  A text based approach works well for sequence diagrams and gets rid of that layer of unnecessary complexity.

Filed under: Awesome, DevTools No Comments
11Mar/09Off

Integrating NaturalDocs With SyntaxHighlighter (For The Win!)

Posted by Charles Chen

In working on some SDK-style developer documentation for FirstPoint, it occurred to me that we needed a way to create some all encompasing documentation which covered not only our code base, but also our markup, our JavaScript controls, CSS, and so on. We currently have most of this stuff in Trac wiki pages, which is a great place to put them, but our Trac deployment is going away and being replaced by Jira...or so I'm told.

In light of this, we needed a way to create portable, useful, developer documentation which included a mix of some auto-generated content and hand crafted documents as well (how-to's and stuff, which would be really terse if placed in code comment). There aren't really any do-it-all tools, but I stumbled across NaturalDocs which seemed to be the most well rounded tool of the ones I looked into (i.e. JSDocs, YUI Doc, a few others - it started off as a search for a tool for documenting UI conventions, markup, and script usage for our team) because of the fact that it allowed for the inclusion of loose .txt files which would essentially be treated like wiki pages.

You can see an example of the output at the MapQuest API documentation site.

One of the main reasons I liked NaturalDocs is because of the support for code blocks and how easy it is to write them in the loose text files. However, the downside is that the generated output is pretty...boring. Here's an example:

You can see that it has no intelligence with regards to the language. Doing a little digging around the 'Net, I found a ticket for a request for support for syntax highlighting. So I ended up rolling my sleeves up and solving this myself. I decided to integrate against the SyntaxHighlighter JavaScript library since I've used it previously and I like the output :-) .

Here is the end result:

I hadn't touched Perl in quite some time (since college), so I had to dig around in there for a bit but I was able to integrate it after a few hours of flailing.

The steps required are as follows (these steps assume you use framed mode):

In the file FramedHTML.pm, you will need to add the following lines to the method BuildFile after the call to $self->ClosingBrowserStyles():

. '<script language="javascript" src="' . $self->MakeRelativeURL($outputFile, $self->HighlighterShCore(), 1) . '"></script>'
. '<script language="javascript" src="' . $self->MakeRelativeURL($outputFile, $self->HighlighterShBrushCSharp(), 1) . '"></script>'
. '<script language="javascript" src="' . $self->MakeRelativeURL($outputFile, $self->HighlighterShBrushXml(), 1) . '"></script>'
. '<script language="javascript" src="' . $self->MakeRelativeURL($outputFile, $self->HighlighterShBrushCss(), 1) . '"></script>'
. '<script language="javascript" src="' . $self->MakeRelativeURL($outputFile, $self->HighlighterShBrushJs(), 1) . '"></script>'
. '<script language="javascript" src="' . $self->MakeRelativeURL($outputFile, $self->HighlighterShBrushSql(), 1) . '"></script>'
. '<script language="javascript">'
. 'SyntaxHighlighter.config.clipboardSwf = "' . $self->MakeRelativeURL($outputFile, $self->HighlighterClipboard(), 1) . '";'
. 'SyntaxHighlighter.all();'
. '</script>'

Next, to support the new getters, you will need to modify HTMLBase.pm and add the following lines:

sub HighlighterShCore
	{
	my $self = shift;
	return NaturalDocs::File->JoinPaths($self->JavaScriptDirectory(), 'shCore.js' );
	};

sub HighlighterShBrushCSharp
	{
	my $self = shift;
	return NaturalDocs::File->JoinPaths($self->JavaScriptDirectory(), 'shBrushCSharp.js' );
	};

sub HighlighterShBrushXml
	{
	my $self = shift;
	return NaturalDocs::File->JoinPaths($self->JavaScriptDirectory(), 'shBrushXml.js' );
	};

sub HighlighterClipboard
	{
	my $self = shift;
	return NaturalDocs::File->JoinPaths($self->JavaScriptDirectory(), 'clipboard.swf' );
	};

sub HighlighterShBrushCss
	{
	my $self = shift;
	return NaturalDocs::File->JoinPaths($self->JavaScriptDirectory(), 'shBrushCss.js' );
	};

sub HighlighterShBrushJs
	{
	my $self = shift;
	return NaturalDocs::File->JoinPaths($self->JavaScriptDirectory(), 'shBrushJScript.js' );
	};

sub HighlighterShBrushSql
	{
	my $self = shift;
	return NaturalDocs::File->JoinPaths($self->JavaScriptDirectory(), 'shBrushSql.js' );};

You should add more to handle whatever syntaxes you need to handle.

By default, the code blocks are generated as <blockquote><pre></pre></blockquote>. To support SyntaxHighlighter, we'll need to change this to allow customizing the class name on the <pre> tag. I decided to use the suggested syntax for including this in the markup: (start code <language>). For example: (start code js). The first module that we have to modify to support this is Native.pm. In the method FormatBody, I made the following change to support the extra token:

# If the line looks like a code tag...
# [CHUCK] ORIGINAL: elsif ($commentLines->[$index] =~ /^\( *(?:(?:start|begin)? +)?(?:table|code|example|diagram) *\)$/i)
elsif ($commentLines->[$index] =~ /^\( *(?:(?:start|begin)? +)?(?:table|code|example|diagram) *((?:\w+)?) *\)$/i)
	{
	if (defined $textBlock)
		{
		$output .= $self->RichFormatTextBlock($textBlock);
		$textBlock = undef;
		};
	# [CHUCK] ORIGINAL: $output .= $tagEnders{$topLevelTag} . '<code>';
	$output .= $tagEnders{$topLevelTag} . "<code class=$1>";
	$topLevelTag = TAG_TAGCODE;
	}

You can see that I've introduced a capturing group to the regular expression to grab the language type (matching SyntaxHighlighter's language strings). The next step is to modify the generation of the intermediate <code> tag to include a class attribute.

If you stop here, the output generation doesn't work correctly since this only affects the intermediate output. We need to jump to HTMLBase.pm and modify the method NDMarkupToHTML so that we can generate the proper tag structure. Here are my modifications:

# [CHUCK] ORIGINAL: my @splitText = split(/(<\/?code>)/, $text);
my @splitText = split(/(<\/?code *(?:class=[^\>]+)?>)/, $text);

while (scalar @splitText)
	{
	$text = shift @splitText;

	if ($text =~ m/^(?:<code *(?:class=([^\>]*))?>)$/i)
		{
		$output .= "<blockquote><pre class=\"brush: $1\">";
		$inCode = 1;
		}

You can see that here, I changed the regular expression used to split the intermediate output into chunks to properly split on the new markup structure. In the if-statement, I changed the eq comparison to a regular expressoin match with a capturing group and inserted that into the output <pre> tag (for the win!).

Now be warned: this is not a complete fix. While this addresses the major issue, generation of the proper output, I did not make changes to copy the image files and JavaScript files required by SyntaxHighlighter (sorry, you're going to have to do that yourself :-P ). The next set of steps are to:

  1. Copy the images associated with SyntaxHighlighter to the \output\styles directory (or whereever you like).
  2. Copy the scripts required for SyntaxHighlighter to the \output\javascript directory.
  3. Modify the paths in the CSS for the icons used with SyntaxHighlighter (in the shCore.css file).

Of course, the themes and CSS files are easy to include since you can specify those at the command line.

That's all there is to it! Happy documenting! I've attached sample files (including the modified source) for SyntaxHighlighter 1.5 and 2.0.

natural-doc-sh2.0.7z (315.01 KB)
natural-doc-sh1.5.7z (281.7 KB)

Filed under: Awesome, Dev, DevTools 2 Comments
   Older Entries »

Me: Charles Chen

Useless Tweets

View more tweets

Recent Comments

Recent Posts

Calendar

May 2012
M T W T F S S
« Apr    
 123456
78910111213
14151617181920
21222324252627
28293031  

Blogroll

Meta

Categories

Archives