<CharlieDigital/> Programming, Politics, and uhh…pineapples

29Jul/11Off

Working with GUIDs in MongoDB and ASP.NET MVC3

Posted by Charles Chen

Just a small tip for those looking to use GUIDs as document IDs in MongoDB in conjunction with ASP.NET MVC3: it's a lot more straightforward than it may seem at the onset.

These examples are based off of the ASP.NET MVC3 tutorials...except with MongoDB instead of EF+SQL Server.

I've set up my model class like so:

public class Movie
{
    [BsonId]
    public Guid ID { get; set; }
    public string Title { get; set; }
    public DateTime ReleaseDate { get; set; }
    public string Genre { get; set; }
    public decimal Price { get; set; }
}

When the application creates an object and persists it to the database, you'll see that it shows up like this in the Mongo console (I've formatted the JSON for clarity):

> db.movies.find()
{
   "_id":BinData(3,"n2FLBkAkhEOCkX42BGXRqg=="),
   "Title":"Test",
   "ReleaseDate":   ISODate("2011-05-11T04:00:00   Z"),
   "Genre":"Comedy",
   "Price":"9.99"
}

If you try to serialize this to JSON, instead of getting a GUID string, you'll get:

// Get a document
BsonDocument document = movies.FindOneAs<BsonDocument>();

// Direct to JSON
document.ToJson();
/*
{
   "_id":new BinData(3,
   "n2FLBkAkhEOCkX42BGXRqg=="   ),
   "Title":"Test",
   "ReleaseDate":   ISODate("2011-05-11T04:00:00   Z"),
   "Genre":"Comedy",
   "Price":"9.99"
}
*/

// With settings
JsonWriterSettings settings = new JsonWriterSettings{OutputMode = JsonOutputMode.JavaScript };
document.ToJson(settings);
/*
{
   "_id":{
      "$binary":"n2FLBkAkhEOCkX42BGXRqg==",
      "$type":"03"
   },
   "Title":"Test",
   "ReleaseDate":Date(1305086400000),
   "Genre":"Comedy",
   "Price":"9.99"
}
*/

This is somewhat inconvenient if you want to work with it from a pure JavaScript perspective; I was hoping that it would have returned a GUID as a string instead.  I was also concerned that this meant that I'd have to manage this manually as well on the server side in my actions, but it turns out that it works better than expected.  The only caveat is that you have to use "_id" when creating queries; otherwise, you can use the GUID as-is and the Mongo APIs will convert it behind the scenes:

public ActionResult Details(Guid id)
{
    MongoCollection<Movie> movies = _database.GetCollection<Movie>("movies");

    // No need to mess with the GUID; use it as is.
    Movie movie = movies.FindOneAs<Movie>(Query.EQ("_id", id)); 

    return View(movie);
}

You can see the result below in the browser:

Note the properly formatted GUID in the URL

So far, so good with my little Mongo+MVC3 experiment :D

Filed under: .Net, Mongo, MVC No Comments
25Jul/11Off

jQuery parsererror and WCF Data Services

Posted by Charles Chen

If you're using WCF Data Services (REST APIs for SharePoint, for example) and you're getting the mysterious "parsererror" error message from jQuery, chances are you'll need to modify your scripts according to this bug report.

The root of the error is the occurrence of single quotes within your JSON response.  This can be fixed by adding the following snippet of code before you make your AJAX calls:

$.ajaxSetup({
    // use custom converter to handle invalid json data
    // returned by WCF Data Service
    // fixes invalid escaped single quotes (\') in json data
    // e.g. { "foo": "bar\'tender" } --> { "foo": "bar'tender" }
    converters: {
        "text json": function( textValue ) {
            return jQuery.parseJSON( textValue.replace(/(^|[^\\])\\'/g, "$1'") );
        }
    }
});

Oddly, this error seemingly came out of nowhere for me; script was working fine one day and broken the next...

Luckily, the patch in this ticket seems to have fixed it for me.

Filed under: jQuery, SharePoint No Comments
18Jul/11Off

Powershell, SharePoint Solutions, and Waiting for Deployment

Posted by Charles Chen

When installing SharePoint solution packages using Powershell, you'll need to wait for the solution to be installed after calling install-spsolution before you can enable features using enable-spfeature.  As a part of an automated script, you'd like for the script to know when the install is finished before continuing.

It's surprisingly easy and can be accomplished with only a tiny bit of code:

$sln = add-spsolution [YOUR_SOLUTION_PATH_HERE]

install-spsolution -identity [YOUR_SOLUTION_NAME_HERE] -gacdeployment

while($sln.JobExists)
{
    echo "Deployment in progress..."

    start-sleep -s 5
}

That's it!

On uninstall, the process is a little bit different as uninstall-spsolution does not return an instance of the solution.  You will want to grab the solution before you start the retraction:

$sln = get-spsolution -identity [YOUR_SOLUTION_IDENTITY]

uninstall-spsolution -identity [YOUR_SOLUTION_IDENTITY] -confirm:$false -webapplication:$webAppName

echo "Started solution retraction..."

while($sln.JobExists)
{
	echo " > Uninstall in progress..."
	start-sleep -s 5
}

remove-spsolution -identity [YOUR_SOLUTION_IDENTITY] -confirm:$false

This helps make for a much smoother install and uninstall process as you can be more certain that your solution is installed before executing code dependent on the solution (feature activation, solution removal, etc).

Filed under: SharePoint No Comments
15Jul/11Off

Chaining jQuery AJAX Calls (w/o Plugins)

Posted by Charles Chen

Here's the scenario: you need to make a series of AJAX calls to process a list of objects and each call is dependent on the results from the previous call.  How can we structure this elegantly in jQuery without having to write massive chains or script callbacks?

An example would be using an AJAX based API to create a hierarchy of nested folders.  Perhaps the user enters a string like "/path1/path2/path3" and multiple calls are needed to create "/path1", then /path1/path2," then "/path1/path2/path3".

My friend John Peterson brought up jQuery deferred to me today and it clicked and my life is better for it.

Here's an example:

var parts = ["path1","path2","path3"]; // Output from string.split()
var chain = $.Deferred(); // Create the root of the chain.
var promise; // Placeholder for the promise

// Build the chain
for(var i = 0; i < parts.length; i++)
{
    if(i == 0) promise = chain;

    // Pipe the response to the "next" function
    promise = promise.pipe(function(response)
    {
        var part = this.shift(); // Get the current part

        var root = "";

        if(response)
        {
            root = response.newPath;
        }

        return $.ajax // MUST call return here.
        ({
           type: "GET",
           url: "filemanager/create/" + root + part,
           context: this
        });
    })
}

promise.done(function(response){
    // This handles the response from the final AJAX call
});

chain.resolveWith(parts); // Execute the chain

The general idea is that we start with a chain and "pipe" the output from each step to the next.  When I execute the chain, I pass in the array as the argument on line 35.

This is necessary because referring to part[i] inside the function declared on line 11 will always yield "part3" .  To work around this, we pass in the whole array of parts as the context to the chain.  It becomes the "this" reference and we simply shift() a value off of the array to get the "current" item.

To pass the array to the next function in the chain, we simply set the context of the AJAX call to the "this" reference.  Now, in the next handler in the chain, the "response" is the output of the previous AJAX call and the "this" reference is the "current" item in the array.

Of course, the final call -- if you want to handle it -- can be added to the chain using done() instead of pipe().

It's a brilliant way of chaining multiple jQuery AJAX calls in a much more coherent manner!

Filed under: Awesome, Dev, jQuery No Comments
14Jul/11Off

WCF Data Services Cheat Sheet

Posted by Charles Chen

I was working with some SharePoint REST queries and couldn't find a good list of the operations supported by the REST API.  Namely, I was trying to figure out if it supported a "contains" operation (it does using indexof).

I found a very nicely put together set of tables which shows the LINQ queries, the corresponding URL, and the SQL produced by the query.

This is the result of my tiny research about translating LINQ to REST into oData URI Conventions and finally into T-SQL. Event though I knew what I needed on the T-SQL side, I was not sure which LINQ statement would create the respective SQL query. So, as an experimental test bed I created a little console application and used Netflix oData Service to see how LINQ to REST gets translated to URIs

Very useful indeed.

On a related note, for those that are working with the REST APIs for SharePoint, I've found that you can actually reduce your network traffic when using $expand by taking only the fields that you're interested in for the expanded item as well. For example:

http://[site]/_vti_bin/listadata.svc/Tasks?$expand=AssignedTo&$select=AssignedTo/Name,AssignedTo/Account,Id,Title

The downside is that this will barf out invalid JSON if you have a null value for the expanded field (in this case, if the task is not assigned to any user).  Here's a sample output:

{
   "d":{
      "results":[
         {
            "__metadata":{
               "uri":"http://project.dev.com/PWA/ts1/_vti_bin/listdata.svc/Tasks(2)",
               "etag":"W/\"3\"",
               "type":"Microsoft.SharePoint.DataService.TasksItem"
            },
            "Id":2,
            "ContentType":"Task",
            "Title":"Action 1",
            "Path":"/PWA/ts1/Lists/Tasks/Milestone 1",
            "PriorityValue":"(2) Normal",
            "StatusValue":"Not Started",
            "AssignedTo":{
               "__metadata":{
                  "uri":"http://project.dev.com/PWA/ts1/_vti_bin/listdata.svc/UserInformationList(18)",
                  "etag":"W/\"7\"",
                  "type":"Microsoft.SharePoint.DataService.UserInformationListItem"
               },
               "Name":"Charles Chen",
               "Account":"DEV\\charles"
            },
            "StartDate":"\/Date(1310428800000)\/",
            "DueDate":"\/Date(1310601600000)\/"
         },
         {
            "error":{
               "code":"",
               "message":{
                  "lang":"en-US",
                  "value":"An error occurred while processing this request."
               }
            }
         }

You can see that the second item in the "results" array is where the task was not assigned to a user.  The response is abruptly cut off with the JSON being completely broken at that point and in an invalid state.  I would have expected that the result would just set "AssignedTo":null

A bummer, but it doesn't detract from my love for the REST APIs :-D

Filed under: Awesome, SharePoint, WCF 1 Comment
5Jul/11Off

The Beauty of Mongo

Posted by Charles Chen

MongoDB is sexy.  Sexy as hell.

This isn't the first or the best intro to Mongo+.NET, but I hope this one can show you how it finally solves the object-relational disconnect with an easy to follow example.

Let's start with the model:

#region prologue

// Charles Chen

#endregion

using System;
using System.Collections.Generic;
using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;

namespace MongoAssessmentsTest
{
    public class Assessment
    {
        private DateTime _created;
        private string _description;
        [BsonId]
        private ObjectId _id;
        private List<ContentItem> _items;
        private DateTime _lastUpdated;
        private string _ownerId;
        private List<string> _tags;
        private string _title;

        public ObjectId Id
        {
            get { return _id; }
            set { _id = value; }
        }

        public string OwnerId
        {
            get { return _ownerId; }
            set { _ownerId = value; }
        }

        public string Title
        {
            get { return _title; }
            set { _title = value; }
        }

        public string Description
        {
            get { return _description; }
            set { _description = value; }
        }

        public List<string> Tags
        {
            get { return _tags; }
            set { _tags = value; }
        }

        public DateTime Created
        {
            get { return _created; }
            set { _created = value; }
        }

        public DateTime LastUpdated
        {
            get { return _lastUpdated; }
            set { _lastUpdated = value; }
        }

        public List<ContentItem> Items
        {
            get { return _items; }
            set { _items = value; }
        }
    }

    public class ContentItem
    {
        private List<Question> _questions;
        private string _text;

        public string Text
        {
            get { return _text; }
            set { _text = value; }
        }

        public List<Question> Questions
        {
            get { return _questions; }
            set { _questions = value; }
        }
    }

    [BsonKnownTypes(typeof(CheckboxQuestion), typeof(RadioQuestion), typeof(SelectQuestion), typeof(FreeTextQuestion))]
    public class Question
    {
        private int _order;
        private int _points;
        private string _text;

        public string Text
        {
            get { return _text; }
            set { _text = value; }
        }

        public int Order
        {
            get { return _order; }
            set { _order = value; }
        }

        public int Points
        {
            get { return _points; }
            set { _points = value; }
        }
    }

    public class CheckboxQuestion : Question
    {
        private List<string> _answers;
        private List<string> _choices;

        public List<string> Choices
        {
            get { return _choices; }
            set { _choices = value; }
        }

        public List<string> Answers
        {
            get { return _answers; }
            set { _answers = value; }
        }
    }

    public class SelectQuestion : Question
    {
        private string _answer;
        private List<string> _choices;

        public List<string> Choices
        {
            get { return _choices; }
            set { _choices = value; }
        }

        public string Answer
        {
            get { return _answer; }
            set { _answer = value; }
        }
    }

    public class RadioQuestion : SelectQuestion { }

    public class FreeTextQuestion : Question
    {
        private List<string> _acceptableAnswers;
        private string _isAnswerCaseSensitive;

        public string IsAnswerCaseSensitive
        {
            get { return _isAnswerCaseSensitive; }
            set { _isAnswerCaseSensitive = value; }
        }

        public List<string> AcceptableAnswers
        {
            get { return _acceptableAnswers; }
            set { _acceptableAnswers = value; }
        }
    }
}

The beauty of it is the absolute simplicity of working with this model.  Aside from a few attributes, there's not much thought that needs to be given to collections and inheritance hierarchies (though there are additional attributes and classes that can be used to control how these are stored if so desired).  I love it because this approach keeps your domain models clean.

How do we interact with this model?

#region prologue

// Charles Chen

#endregion

using System;
using System.Collections.Generic;
using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Driver.Builders;

namespace MongoAssessmentsTest
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            Program program = new Program();
            program.Run();
        }

        private void Run()
        {
            MongoServer server = MongoServer.Create(); // Uses default connection options to localhost

            MongoDatabase database = server.GetDatabase("assessments_db");

            if (!database.CollectionExists("assessments"))
            {
                CommandResult result = database.CreateCollection("assessments");

                if (!result.Ok)
                {
                    Console.Out.WriteLine(result.ErrorMessage);
                    return;
                }
            }

            MongoCollection<Assessment> assessments = database.GetCollection<Assessment>("assessments");

            if (assessments.FindOne(Query.EQ("Title", "Sample 1")) == null)
            {
                // Insert an assessment.
                Assessment a = new Assessment
                {
                    Created = DateTime.Now,
                    Description = "A sample assessment",
                    Title = "Sample 1",
                    Items = new List<ContentItem>
                    {
                        new ContentItem
                        {
                            Questions = new List<Question>
                            {
                                new SelectQuestion
                                {
                                    Text = "Who was the first president of the United States?",
                                    Answer = "George Washington",
                                    Choices = new List<string>
                                    {
                                        "George Washington",
                                        "Abraham Lincoln",
                                        "John Adams"
                                    }
                                },
                                new CheckboxQuestion
                                {
                                    Text = "Which of these is NOT a former US President?",
                                    Answers = new List<string>
                                    {
                                        "Benjamin Franklin",
                                        "Hillary Clinton"
                                    },
                                    Choices = new List<string>
                                    {
                                        "Benjamin Franklin",
                                        "Bill Clinton",
                                        "Hillary Clinton",
                                        "Andrew Jackson",
                                        "James Garfield"
                                    }
                                }
                            }
                        }
                    }
                };

                assessments.Insert(a);
            }

            // Get it as BSON - great for writing straight to a web page
            BsonDocument a1 = assessments.FindOneAs<BsonDocument>(Query.EQ("Title", "Sample 1"));

            Console.Out.WriteLine(a1);

            // Get it as an object - great if you want to work with it on the server.
            Assessment a2 = assessments.FindOneAs<Assessment>(Query.EQ("Title", "Sample 1"));

            Console.Out.WriteLine(a2.Title);
        }
    }
}

Brilliantly simple.  Just fire up mongod.exe and you're ready to rock and roll.  Download the example from here: MongoAssessmentsTest.zip

Filed under: .Net, Dev, Mongo No Comments
5Jul/11Off

When is InfoPath Ever the Answer?

Posted by Charles Chen

Since I've started working with SharePoint back in beta 2 of 2007, I've wondered what the infatuation with InfoPath is.

Everyone from enterprise architects and business users seems to be very intrigued by it.

As a more technical guy who's had to dig into it, I find it a puzzling beast and really don't understand why folks even give it a second whiff.

To understand why, let's take a step back and examine the landscape of forms technologies which are available in the typical enterprise environment and how they fit into SharePoint.

Web Forms

  • Pros
    • Extremely flexible and powerful; a developer can create any form imaginable.
    • People are familiar with web forms because they use them: Every.  Single. Day.  When they log on to their gmail, when they buy something from Amazon, when they sign up for Facebook, when they log into The New York Times website -- people use web forms every day.
    • Deploys fairly easily into a SharePoint environment using .wsps and can be permissioned like other SharePoint artifacts.
    • Easily understood by developers and easy to hire developers or contractors to manage; web forms developers are a dime a dozen.
    • Easy to integrate additional data into the forms since it's easy to call out to web services, look up files in a file system, perform searches across SharePoint, etc. to incorporate additional data and intelligence into your forms.
  • Cons
    • Requires a developer to create and update the form.
    • Requires custom programming.
    • Not available on the desktop or offline.
    • Doesn't work well for forms that need to also support print versions since it's nearly impossible to create a printed layout that will 100% match the on screen layout.

Word Document Forms

  • Pros
    • Fairly flexible with a programming model that can be easily accessed by power users via record macro or custom written macros.  Can build fairly complex interactions.
    • Deploys easily into a SharePoint environment by simply uploading a document template to a list and associating it as the default template.
    • Quick Parts map fields directly onto content type fields on the list - save the document back to SharePoint, and the fields are updated in SharePoint.  Update fields in SharePoint and they are pushed down to the Word document.
    • Business users can create these easily.  In fact, if you asked someone in HR -- for example -- to create a form, more likely than not, they will create a Word or Excel based form.
    • Form can be completed offline and saved back to SharePoint.
    • Printed form is pretty much identical to electronic form.
    • Businesses already have large inventories of Word forms.
  • Cons
    • Macro programming isn't accessible to most business users.
    • Macros may present a security risk.
    • Not as easy to perform complex tasks (for example, multiple web service calls) as it is in web forms.
    • No direct integration with the server environment and the SharePoint content store - may still need event receivers and what not to accomplish certain tasks.
    • Requires Word runtime on the desktop.

PDF Forms

  • Pros
    • Form is "locked down" once deployed.
    • Form is easy to build using Adobe Acrobat.
    • Printed form is identical to electronic form; create one form and use it everywhere.  This is especially useful in some cases in some industries like life sciences.
    • Everyone has Acrobat on their desktops.
    • Business users already have the skills to work with Adobe PDF forms and PDF forms can be easily created from Word forms.
    • Businesses already have large inventories of PDF forms.
  • Cons
    • Requires paid versions of Acrobat to create forms, but your business users probably already have these.
    • Requires some custom programming to expose the form fields to SharePoint (but easy to program and only needs to be done once); not supported out of the box.
    • Dynamic forms are difficult to create, requiring developers to write embedded Javascript or logic to build the form dynamically on the server before serving it.

InfoPath Forms

  • Pros
    • Microsoft really wants you to use it and thus they've rigged it into various points in SharePoint, Office, and SharePoint workflows (but you can do these with web forms, too).
    • Some forms can be used in the web or on the desktop.
    • Easy to build basic forms (but not as easy as it is to use Quick Parts in Word).
  • Cons
    • Requires license to InfoPath to create forms and your business users probably don't have these.
    • If you ask a business user to create a form, the user will either: 1) create it in Word, 2) create it in Excel, 3) create it in PDF, 4) ask IT to create it.  No business user will instinctively think "Ah, I'll create an InfoPath form".  It's not a technology easily leveraged by anyone but someone technical.  The idea that power users can create and own their forms is a myth; no business user wants to spend their week working on a form when they have hamsters in IT to do that for them.
    • With that said, it's going to be a LOT easier to find developers who are strong in web forms (and they'll probably be cheaper) than it is to find developers who are strong in InfoPath (and they'll probably require higher rates).
    • While some forms can be used via Forms Server or on the desktop, the truth of the matter is that Forms Server enabled forms are limited to a subset of the desktop forms' functionality.
    • InfoPath forms are difficult to work with programmatically compared to writing Word macros or ASP.NET web forms.
    • Anything beyond basic interactions will require programming anyways, so why not pick the technology that's easier to program for?
    • Complex form fields (for example, repeating rows) don't map to SharePoint fields and cannot be surfaced in SharePoint as metadata, making it somewhat less useful unless you're willing to spend the effort to manually manage that serialization of data.  But then, why would you choose InfoPath and not Word or web or PDF forms?
    • Electronic forms are not identical to printed versions (as is required for some FDA forms which can be submitted electronically or as paper forms or as scanned versions of paper forms) so in some workflows, you'll need to create separate forms for printed versions and consolidate that data manually.  If you need to create those printed forms in Word or PDF anyways, why bother creating InfoPath forms?
    • Difficult to share forms with users outside of your corporate firewall because: 1) they probably won't have InfoPath on their desktop (but they will have or can easily get Acrobat or Word) or 2) you have to get them access to your network to serve them the form via Forms Server.  With Word based forms or PDF forms, you could theoretically just email the forms, collect them, and upload them -- it gives you the flexibility to create these types of workflows.

Ultimately, I think InfoPath is a poor choice for forms because there are better options either from the perspective of developers or the perspective of a business user.  It's a myth that InfoPath will allow business users to self service and easily create and customize their own forms and thus save time and money.  If anything, it's even easier for business users to create their forms in Word, a tool they are already intimately familiar with.  Anything beyond the most basic of forms will require programming and IT involvement anyways and if that's the case, why bother with InfoPath when it's harder to resource for?

I don't think I'll ever understand the infatuation with InfoPath as a solution.  I do wonder if most folks are simply unaware of the Quick Parts functionality in Word 2007 and 2010 and how nicely it integrates with SharePoint.

Filed under: Rants No Comments
4Jul/11Off

Straight Talk from Fareed Zakaria

Posted by Charles Chen

An excellent interview from NPR with Fareed Zakaria.

Straight talk about the problems that the US is facing and the straightforward solutions that are being defeated by politics and ideology instead of pragmatism.

No better way to spend 45 minutes today.

1Jul/11Off

Thoughts on MongoDB

Posted by Charles Chen

I just recently posted a short write up on Fluent NHibernate vs. Code First in Enterprise Library 4.1.

I'm starting up a new pet project (or rather re-starting it after a 3 year hiatus) so I've been trying to figure out the underlying technologies that I want to build on (as I mentioned, I've been in the SharePoint space for so long now, that I haven't been up-to-date on the latest developments in DB storage).  To be honest, I wasn't really satisfied with the experience of either FNH or EF4.1, especially as I started to work with my model more and more.

What I realized is that the object-relational disconnect is still there.  I'm starting to think that no amount of lipstick will make that pig presentable.   It doesn't make a whole lot of sense and is an impediment to productivity in today's application-layer-oriented environment (if that makes any sense...); I really, really, really want to stop building database applications and focus my time and energy on the application layer.

Enter MongoDB.  I think this (and the similarly JavaScript-friendly, document-oriented, seemingly less performant CouchDB) are what I've been yearning for in the data storage space.  First, I love JavaScript; it's my favorite language (despite whatever flaws it has) for its flexibility and dynamic nature.   So the fact that Mongo's interface language is JavaScript and that it's data storage format is a form of JSON is really appealing to me.  Second, the document-oriented nature of the storage model -- combined with the JSON storage format -- really goes a long way in terms of solving the object-relational disconnect because, well, the relational part is gone.  You have objects and embedded objects and they map perfectly to JSON objects which in turn map fairly easily into .NET CLR objects.

Of course, one neat thing about the nature of working with JSON from UI to the database is that in some cases, the application layer can be relegated to a relatively dumb pass-through (seems like it's still needed as one area of weakness for Mongo is authentication and authorization).  Get your objects in JSON, bind to them in the browser using KO, modify them, and send them back!  It's a beautiful paradigm that could only be better with an out-of-the-box REST interface like CouchDB.

I'm just digging in, but I'm starting to think that I'm never going to look back at relational databases outside of client work and possibly as a supplementary data store for reporting purposes.  It only takes a few moments to get started with Mongo so give it a shot!

Filed under: Dev, Mongo No Comments