<CharlieDigital/> Programming, Politics, and uhh…pineapples

18May/11Off

Introducing GameTime – Real Time Collaboration for SharePoint

Posted by Charles Chen

Background

Since 2005, I've been thinking about building a web-based, real-time collaboration solution.  Back in those days, I had just discovered AJAX.NET (before ASP.NET provided an implementation of AJAX) and I had drawn up a design for a chat-centric collaboration platform.  My friend and co-worker Dan Chawner would sit in adjacent cubicles and exchange IMs over MSN Messenger as we worked on projects.  I thought: "wouldn't it be great if I could actually do things with these IMs instead of copying/pasting them?"

Over the years, that design languished as I moved on to other interests.  When I first saw Groove (before Microsoft purchased it) and Wave, it brought back memories of those designs.  Wave, in particular, was very close to what I had imagined building (albeit without all of the crazy in-line edits and what not).  I had kind of given up the idea after not being able to find any direction myself on how to make such a tool useful.

Experience

It turns out what I needed was more experience -- both technically and professionally -- to finally put it all together.  One thing that I've learned in the last few years of working with SharePoint is that it's generally a really cumbersome platform for collaboration when left to it's own devices.  It's great for:

  1. Storing documents
  2. Finding documents you've stored
  3. Storing lists of things
  4. ???

Everything else?  I guess it's kind of mediocre.

And yet, organizations -- multi-billion dollar organizations -- depend on SharePoint as a platform for collaboration, communication, sharing information, and in general, getting things done.

This is what experience has taught me as I sat through scrums watching folks update list items, as I dealt with the deluge of emails sent "Reply All" trying to figure out the status of tasks, and as I dealt with communicating effectively as a part of a team of remote consultants.

There are real inefficiencies when you try to use out-of-the-box SharePoint for scenarios which it was not designed and it's not a terribly useful platform for collaboration so much as it is for storage and retrieval of information (and even some would debate how well it's designed for those purposes....).

The question we set out to answer is how can we make the SharePoint platform more efficient for collaboration?  How can we help teams that work with remote members collaborate and communicate effectively?  How can we make SharePoint more than just a document and information repository?  How can we enable SharePoint to deliver notifications and updates in real-time?

Opportunity

Right before Christmas, my wife was put on strict bed rest at home carrying our daughter, Charlotte (she was deemed a high risk pregnancy as we've lost three other fetuses in two prior pregnancies).  At first, I considered taking the 6 week unpaid family leave.  But our due date was at the end of April; that would hardly get me through February with my vacation days.  I knew I had to quit and tough it out for at least these 4 months to make sure that we carried this baby to term.

This is when I finally put two-and-two together: I had to use this one opportunity to take a risk, go all-in and try to manifest this idea that I've been carrying around with me for years.

The Result

We've been "dogfooding" it for over a month now!

What came out of this process is GameTime, a real-time collaboration solution built on SharePoint and the same underlying technology in Google Wave, XMPP.  In one sentence?  It's Campfire for SharePoint.

At the core of GameTime is the concept of a "Huddle" where team members come together around a web-based chat interface.  But it's more than that; we've integrated it with SharePoint document libraries and lists to create a context for real-time collaborative efforts right in SharePoint.  Each Huddle is composed of collaborators, documents, milestones, and tasks -- the essentials of any collaborative effort and it's all wired up to react in real-time.

When a document is checked out in SharePoint, a real-time notification shows up in the chat stream and the document is updated in the Huddle.  When a new task is created and assigned, a real-time notification shows up in the chat stream and the task is added to the Huddle.  When a user comes into the Huddle, a real-time presence notification is sent and the user's status is updated immediately in the Huddle.

GameTime finally gives SharePoint users an actual reason to be in the SharePoint environment outside of point interactions (for example: trying to find a document); it gives SharePoint a central role in day-to-day collaboration instead of being just a storage repository that is called upon once in a while.  But even more importantly, perhaps, is that it adds a real-time element to SharePoint.  No more waiting for email notifications.  No more playing email-tag to get the status of tasks.  No more waiting for someone to check documents in/out.  You can see SharePoint activity in real-time right from your Huddle.

This short demo video should give you an idea of the functionality and capabilities of the product (this video represents about 60% of the current functionality):

Now the Hard Part

It's taken the small team of John Peterson (and his alter ego "Tyrone Engels") and myself nearly 4 months of work to get GameTime to this point and just this week, we've started our first AdWords campaign -- a great milestone.  The challenge of spreading the word and getting our first sale is now before us so indulge me with this shameless plug!

If your organization runs SharePoint 2010 (Server or Foundation) and you're interested in trying out that real-timey goodness of GameTime, fill out our contact form and get your first 10 licenses, free.  You can also use the form to schedule a live demo in our hosted environment.  I truly believe that you'll be sold once you experience it, live.

The Future

While we're focused on getting our first sale, we've started to plan for upcoming tradeshows and we've started to develop our next set of features.  These include:

  • Higher level, real-time dashboards built off of the same platform
  • Mobile integration for Android, Blackberries, iPhones, and Windows Phone
  • Chat and real-time notifications everywhere in the SharePoint environment -- get immediate notification of changes anywhere you are SharePoint.

So head over to our web site: http://thinktastic.com and contact us to get a fully featured trial license!

Filed under: News, SharePoint, XMPP No Comments
9Jul/09Off

XMPP SASL Challenge-Response Using DIGEST-MD5 In C#

Posted by Charles Chen

I've been struggling mightily with implementing the SASL challenge-response portion of an XMPP client I've been working on. By far, this has been the hardest part to implement as it's been difficult to validate whether I've implemented the algorithm correctly as there doesn't seem to be any (easy to find) open source implementations of of SASL with the DIGEST-MD5 implementation (let alone in C#).

The trickiest part of the whole process is building the response token which gets sent back as a part of the message to the server.

RFC2831 documents the SASL DIGEST-MD5 authentication mechanism as such:

   Let { a, b, ... } be the concatenation of the octet strings a, b, ...
 

   Let H(s) be the 16 octet MD5 hash [RFC 1321] of the octet string s.

   Let KD(k, s) be H({k, ":", s}), i.e., the 16 octet hash of the string
   k, a colon and the string s.

   Let HEX(n) be the representation of the 16 octet MD5 hash n as a
   string of 32 hex digits (with alphabetic characters always in lower
   case, since MD5 is case sensitive).

      response-value  =
         HEX( KD ( HEX(H(A1)),
                 { nonce-value, ":" nc-value, ":",
                   cnonce-value, ":", qop-value, ":", HEX(H(A2)) }))

   If authzid is specified, then A1 is

      A1 = { H( { username-value, ":", realm-value, ":", passwd } ),
           ":", nonce-value, ":", cnonce-value, ":", authzid-value }

   If authzid is not specified, then A1 is

      A1 = { H( { username-value, ":", realm-value, ":", passwd } ),
           ":", nonce-value, ":", cnonce-value }

   where

         passwd   = *OCTET

   If the "qop" directive's value is "auth", then A2 is:

      A2       = { "AUTHENTICATE:", digest-uri-value }

   If the "qop" value is "auth-int" or "auth-conf" then A2 is:

      A2       = { "AUTHENTICATE:", digest-uri-value,
               ":00000000000000000000000000000000" }

Seems simple enough, right? Not! It took a bit of time to parse through it mentally and come up with an impelementation, but I was still failing (miserably).

The breakthrough came when I stumbled upon a posting by Robbie Hanson:

Here's the trick - normally when you hash stuff you get a result in hex values. But we don't want this result as a string of hex values! We need to keep the result as raw data! If you were to do a hex dump of this data you'd find it to be "3a4f5725a748ca945e506e30acd906f0". But remeber, we need to operate on it's raw data, so don't convert it to a string.

The most important part of his posting is the last line (and that it included the intermediate hexadecimal string results. Win! Now I finally had some sample data to compare against to figure out where I was going wrong). At one critical junction in my implementation of the algorithm, I was converting the MD5 hash value to a hexadecimal string -- thank goodness for Robbie's clarification of that point!

Armed with this test data, I was finally able to get it all working. Here is the test code:

using MbUnit.Framework;
using Xmpp.Client;
 

namespace Xmpp.Tests
{
    [TestFixture]
    public class SaslChallengeResponseTests
    {
        [Test]
        public void TestCreateResponse()
        {
            // See example here: http://deusty.blogspot.com/2007/09/example-please.html

            // h1=3a4f5725a748ca945e506e30acd906f0
            // a1Hash=b9709c3cdb60c5fab0a33ebebdd267c4
            // a2Hash=2b09ce6dd013d861f2cb21cc8797a64d
            // respHash=37991b870e0f6cc757ec74c47877472b

            SaslChallenge challenge = new SaslChallenge(
                "md5-sess", "utf-8", "392616736", "auth", "osXstream.local");

            SaslChallengeResponse response = new SaslChallengeResponse(
                challenge, "test", "secret", "05E0A6E7-0B7B-4430-9549-0FE1C244ABAB");

            Assert.AreEqual("3a4f5725a748ca945e506e30acd906f0", 
                response.UserTokenMd5HashHex);
            Assert.AreEqual("b9709c3cdb60c5fab0a33ebebdd267c4", 
                response.A1Md5HashHex);
            Assert.AreEqual("2b09ce6dd013d861f2cb21cc8797a64d", 
                response.A2Md5HashHex);
            Assert.AreEqual("37991b870e0f6cc757ec74c47877472b", 
                response.ResponseTokenMd5HashHex);
        }
    }
}

I modeled the SASL challenge like so:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Reflection;
using System.Text;
 

namespace Xmpp.Client
{
    /// <summary>
    /// Represents a SASL challenge in object code.
    /// </summary>
    public class SaslChallenge
    {
        private static readonly Dictionary<string, FieldInfo> _fields;

        private readonly string _rawDecodedText;
        private string _algorithm;
        private string _charset;
        private string _nonce;
        private string _qop;
        private string _realm;

        /// <summary>
        /// Initializes the <see cref="SaslChallenge"/> class.
        /// </summary>
        /// <remarks>
        /// Caches the properties which are set using reflection on <see cref="Parse"/>.
        /// </remarks>
        static SaslChallenge()
        {
            // Initialize the hash of fields.
            _fields = new Dictionary<string, FieldInfo>();

            FieldInfo[] fields = typeof (SaslChallenge).GetFields(
                BindingFlags.NonPublic | BindingFlags.Instance);

            foreach (FieldInfo field in fields)
            {
                // Trim the _ from the start of the field names.
                string name = field.Name.Trim('_');

                _fields[name] = field;
            }
        }

        /// <summary>
        /// Creates a specific SASL challenge message.
        /// </summary>
        public SaslChallenge(string algorithm, string charset, 
            string nonce, string qop, string realm)
        {
            _algorithm = algorithm;
            _charset = charset;
            _nonce = nonce;
            _qop = qop;
            _realm = realm;

            Debug.WriteLine("algorithm=" + _algorithm);
            Debug.WriteLine("charset=" + _charset);
            Debug.WriteLine("nonce=" + _nonce);
            Debug.WriteLine("qop=" + _qop);
            Debug.WriteLine("realm=" + _realm);
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="SaslChallenge"/> 
        /// class based on the raw decoded text.
        /// </summary>
        /// <remarks>
        /// Use the <see cref="Parse"/> method to create an instance from 
        /// a raw encoded message.
        /// </remarks>
        /// <param name="rawDecodedText">The raw decoded text.</param>
        private SaslChallenge(string rawDecodedText)
        {
            _rawDecodedText = rawDecodedText;

            string[] parts = rawDecodedText.Split(',');

            foreach (string part in parts)
            {
                string[] components = part.Split('=');

                string property = components[0];

                _fields[property].SetValue(this, 
                    components[1].Trim('"'));
            }

            Debug.WriteLine("algorithm=" + _algorithm);
            Debug.WriteLine("charset=" + _charset);
            Debug.WriteLine("nonce=" + _nonce);
            Debug.WriteLine("qop=" + _qop);
            Debug.WriteLine("realm=" + _realm);
        }

        public string Realm
        {
            get { return _realm; }
        }

        public string Nonce
        {
            get { return _nonce; }
        }

        public string Qop
        {
            get { return _qop; }
        }

        public string Charset
        {
            get { return _charset; }
        }

        public string Algorithm
        {
            get { return _algorithm; }
        }

        public string RawDecodedText
        {
            get { return _rawDecodedText; }
        }

        /// <summary>
        /// Parses the specified challenge message.
        /// </summary>
        /// <param name="response">The base64 encoded challenge.</param>
        /// <returns>An instance of <c>SaslChallenge</c> based on 
        /// the message.</returns>
        public static SaslChallenge Parse(string encodedChallenge)
        {
            byte[] bytes = Convert.FromBase64String(encodedChallenge);

            string rawDecodedText = Encoding.ASCII.GetString(bytes);

            return new SaslChallenge(rawDecodedText);
        }
    }
}

And finally, here is the challenge response class which contains the meat of the response building logic:

using System;
using System.Diagnostics;
using System.Security.Cryptography;
using System.Text;
 

namespace Xmpp.Client
{

    /// <summary>
    /// Partial implementation of the SASL authentication protocol 
    /// using the DIGEST-MD5 mechanism.
    /// </summary>
    /// <remarks>
    /// See <see href="http://www.ietf.org/rfc/rfc4422.txt"/> and 
    /// <see href="http://www.ietf.org/rfc/rfc2831.txt"/> for details.
    /// </remarks>
    public class SaslChallengeResponse
    {
        #region fields

        private static readonly Encoding _encoding;
        private static readonly MD5 _md5;
        private readonly SaslChallenge _challenge;
        private readonly string _cnonce;
        private readonly string _decodedContent;
        private readonly string _digestUri;
        private readonly string _encodedContent;
        private readonly string _password;
        private readonly string _realm;
        private readonly string _username;

        private string _a1Md5HashHex;
        private string _a2Md5HashHex;
        private string _responseTokenMd5HashHex;
        private string _userTokenMd5HashHex;

        #endregion

        #region properties

        /// <summary>
        /// Gets the final, base64 encoded content of the challenge response.
        /// </summary>
        /// <value>A base64 encoded string of the response content.</value>
        public string EncodedContent
        {
            get { return _encodedContent; }
        }

        /// <summary>
        /// Gets the unencoded content of the challenge response.
        /// </summary>
        /// <value>The response content in plain text.</value>
        public string DecodedContent
        {
            get { return _decodedContent; }
        }

        /// <summary>
        /// Gets the hexadecimal string representation of the user token MD5 
        /// hash value.
        /// </summary>
        /// <value>The hexadecimal representation of the user token MD5 hash 
        /// value.</value>
        public string UserTokenMd5HashHex
        {
            get { return _userTokenMd5HashHex; }
        }

        /// <summary>
        /// Gets the hexadecimal string representation of the response token 
        /// MD5 hash value.
        /// </summary>
        /// <value>The hexadecimal string representation of the response token 
        /// MD5 hash value.</value>
        public string ResponseTokenMd5HashHex
        {
            get { return _responseTokenMd5HashHex; }
        }

        /// <summary>
        /// Gets the hexadecimal string representation of the A1 MD5 hash 
        /// value (see RFC4422 and RFC2831)
        /// </summary>
        /// <value>The hexadecimal string representation of the A1 MD5 hash 
        /// value (see RFC4422 and RFC2831)</value>
        public string A1Md5HashHex
        {
            get { return _a1Md5HashHex; }
        }

        /// <summary>
        /// Gets the hexadecimal string representation of the A2 MD5 hash 
        /// value (see RFC4422 and RFC2831)
        /// </summary>
        /// <value>The hexadecimal string representation of the A2 MD5 hash 
        /// value (see RFC4422 and RFC2831)</value>
        public string A2Md5HashHex
        {
            get { return _a2Md5HashHex; }
        }

        #endregion

        /// <summary>
        /// Initializes the <see cref="SaslChallengeResponse"/> class.
        /// </summary>
        static SaslChallengeResponse()
        {
            _md5 = MD5.Create();
            _encoding = Encoding.UTF8;
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="SaslChallengeResponse"/> 
        /// class.
        /// </summary>
        /// <param name="challenge">The challenge.</param>
        /// <param name="username">The username.</param>
        /// <param name="password">The password.</param>
        public SaslChallengeResponse(SaslChallenge challenge, 
            string username, string password)
            : this(challenge, username, password, null, null, null)
        {
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="SaslChallengeResponse"/> 
        /// class.
        /// </summary>
        /// <param name="challenge">The challenge.</param>
        /// <param name="username">The username.</param>
        /// <param name="password">The password.</param>
        /// <param name="cnonce">A specific cnonce to use.</param>
        public SaslChallengeResponse(SaslChallenge challenge, string username, 
            string password, string cnonce)
            : this(challenge, username, password, null, null, cnonce)
        {
        }

        /// <summary>
        /// Initializes a new instance of the <see cref="SaslChallengeResponse"/> 
        /// class.
        /// </summary>
        /// <param name="challenge">The challenge.</param>
        /// <param name="username">The username.</param>
        /// <param name="password">The password.</param>
        /// <param name="realm">A specific realm, different from the one in the 
        /// challenge.</param>
        /// <param name="digestUri">The digest URI.</param>
        /// <param name="cnonce">A specific client nonce to use.</param>
        public SaslChallengeResponse(SaslChallenge challenge, string username, 
            string password, string realm, string digestUri, string cnonce)
        {
            _challenge = challenge;
            _username = username;
            _password = password;

            if (string.IsNullOrEmpty(_challenge.Realm))
            {
                _realm = realm;
            }
            else
            {
                _realm = challenge.Realm;
            }

            if (string.IsNullOrEmpty(_realm))
            {
                throw new ArgumentException("No realm was specified.");
            }

            if (string.IsNullOrEmpty(cnonce))
            {
                _cnonce =
                    Guid.NewGuid().ToString().TrimStart('{').TrimEnd('}')
                        .Replace("-", string.Empty).ToLowerInvariant();
            }
            else
            {
                _cnonce = cnonce;
            }

            if (string.IsNullOrEmpty(digestUri))
            {
                _digestUri = string.Format("xmpp/{0}", _challenge.Realm);
            }
            else
            {
                _digestUri = digestUri;
            }

            // Main work here:
            _decodedContent = GetDecodedContent();

            byte[] bytes = _encoding.GetBytes(_decodedContent);

            _encodedContent = Convert.ToBase64String(bytes);
        }

        /// <summary>
        /// Gets the body of the response in a decoded format.
        /// </summary>
        /// <returns>The raw response string.</returns>
        private string GetDecodedContent()
        {
            // Gets the response token according to the algorithm in RFC4422 
            // and RFC2831
            _responseTokenMd5HashHex = GetResponse();

            StringBuilder buffer = new StringBuilder();

            buffer.AppendFormat("username=\"{0}\",", _username);
            buffer.AppendFormat("realm=\"{0}\",", _challenge.Realm);
            buffer.AppendFormat("nonce=\"{0}\",", _challenge.Nonce);
            buffer.AppendFormat("cnonce=\"{0}\",", _cnonce);
            buffer.Append("nc=00000001,qop=auth,");
            buffer.AppendFormat("digest-uri=\"{0}\",", _digestUri);
            buffer.AppendFormat("response={0},", _responseTokenMd5HashHex);
            buffer.Append("charset=utf-8");

            return buffer.ToString();
        }

        /// <summary>
        /// HEX( KD ( HEX(H(A1)), { nonce-value, ":" nc-value, ":", cnonce-value, 
        /// ":", qop-value, ":", HEX(H(A2)) }))
        /// </summary>
        private string GetResponse()
        {
            byte[] a1 = GetA1();
            string a2 = GetA2();

            Debug.WriteLine("a1=" + ConvertToBase16String(a1));
            Debug.WriteLine("a2=" + a2);

            byte[] a2Bytes = _encoding.GetBytes(a2);

            byte[] a1Hash = _md5.ComputeHash(a1);
            byte[] a2Hash = _md5.ComputeHash(a2Bytes);

            _a1Md5HashHex = ConvertToBase16String(a1Hash);
            _a2Md5HashHex = ConvertToBase16String(a2Hash);

            // Let KD(k, s) be H({k, ":", s})
            string kdString = string.Format("{0}:{1}:{2}:{3}:{4}:{5}",
                _a1Md5HashHex, _challenge.Nonce, "00000001",
                _cnonce, "auth", _a2Md5HashHex);

            Debug.WriteLine("kd=" + kdString);

            byte[] kdBytes = _encoding.GetBytes(kdString);

            byte[] kd = _md5.ComputeHash(kdBytes);
            string kdBase16 = ConvertToBase16String(kd);

            return kdBase16;
        }

        /// <summary>
        /// A1 = { H( { username-value, ":", realm-value, ":", passwd } ), 
        /// ":", nonce-value, ":", cnonce-value }
        /// </summary>
        private byte[] GetA1()
        {
            string userToken = string.Format("{0}:{1}:{2}",
                                             _username, _realm, _password);

            Debug.WriteLine("userToken=" + userToken);

            byte[] bytes = _encoding.GetBytes(userToken);

            byte[] md5Hash = _md5.ComputeHash(bytes);

            // Use this for validation purposes from unit testing.
            _userTokenMd5HashHex = ConvertToBase16String(md5Hash);

            string nonces = string.Format(":{0}:{1}",
                                          _challenge.Nonce, _cnonce);

            byte[] nonceBytes = _encoding.GetBytes(nonces);

            byte[] result = new byte[md5Hash.Length + nonceBytes.Length];

            md5Hash.CopyTo(result, 0);
            nonceBytes.CopyTo(result, md5Hash.Length);

            return result;
        }

        /// <summary>
        /// A2 = { "AUTHENTICATE:", digest-uri-value }
        /// </summary>
        private string GetA2()
        {
            string result = string.Format("AUTHENTICATE:{0}", _digestUri);

            return result;
        }

        /// <summary>
        /// Converts a byte array to a base16 string.
        /// </summary>
        /// <param name="bytes">The bytes to convert.</param>
        /// <returns>The hexadecimal string representation of the contents 
        /// of the byte array.</returns>
        private string ConvertToBase16String(byte[] bytes)
        {
            StringBuilder buffer = new StringBuilder();

            foreach (byte b in bytes)
            {
                string s = Convert.ToString(b, 16).PadLeft(2, '0');

                buffer.Append(s);
            }

            Debug.WriteLine(string.Format("Converted {0} bytes", 
                bytes.Length));

            string result = buffer.ToString();

            return result;
        }
    }
}

Happy DIGESTing!

Filed under: .Net, XMPP 2 Comments