<CharlieDigital/> Programming, Politics, and uhh…pineapples

26Nov/13Off

Preventing the Garbage Collector From Ruining Your Day

Posted by Charles Chen

If you're working with ZeroMQ, you may run into an exception with the message "Context was terminated".

It turns out that this is due to the garbage collector cleaning up (or attempting to clean up?) the ZmqContext.

Found this out via this handy thread on Stack, but what about cases where you can't use a using statement?

For example, in a Windows Service, I create the context on the OnStart method and destroy the context on the OnStop method.

In this case, an alternative is to use the GC.KeepAlive(Object obj) method to prevent the garbage collector from collecting the object until after the call to this method.  It seems counter intuitive, but it is actually a signal to tell the garbage collector that it can collect the object at any point after this call.

Filed under: .Net, Self Note, ZeroMQ No Comments
12Nov/13Off

An Architecture for High-Throughput Concurrent Web Request Processing

Posted by Charles Chen

I've been working with ZeroMQ lately and I think I've fallen in love.

It's rare that a technology or framework just jumps out at you, but here is one that will get your head spinning on the different ways that it can make your architecture more scalable, more powerful, and all the while offering a frictionless way of achieving this.

I've been building distributed, multi-threaded applications since college, and ZeroMQ has changed everything for me.

It initially started with a need to build a distributed event processing engine.  I had wanted to try implementing it in WCF using peer-to-peer and/or MSMQ endpoints, but the thought of the complexity of managing that stack along with the configuration and setup seemed like it would be at least fruitful to look into a few other alternatives.

RabbitMQ and ZeroMQ were the clear front-runners for me.  I really liked the richness of documentation and examples with RabbitMQ and if you look at some statistics, it has a much greater rate of mentions on Stack so we can assume that it has a higher rate of adoption.  But at the core of it, I think that there really is no comparison between these two except for the fact that they both have "MQ" in their names.

It's true that one could build RabbitMQ like functionality on top of ZeroMQ, but to a degree, I think that would be defeating the purpose.  The beauty of ZeroMQ is that it's so lightweight and so fast that it's really hard to believe; there's just one reference to add to your project.  No central server to configure.  No single point of failure.  No configuration files.   No need to think about failovers and clustering.  Nothing.  Just plug and go.  But there is a cost to this: a huge tradeoff in some of the higher level features that -- if you want -- you have to build yourself.

If you understand your use cases and you understand the limitations of ZeroMQ and where it's best used, you can find some amazing ways to leverage it to make your applications more scalable.

One such use case I've been thinking about is using it to build a highly scalable web-request processing engine which would allow scaling by adding lots of cheap, heterogeneous nodes.  You see, with ASP.NET, unless you explicitly build a concurrency-oriented application, your web server processing is single-threaded per request and you can only ever generate output HTML at the sum of the costs of generating each sub part of your view.  To get around this, we could consider a processing engine that would be able to parse controls and send the processing off -- in parallel -- to multiple processors and then reassemble the output HTML before feeding it back to the client.  In this scenario, the cost of rendering the page is the overhead of the request plus the cost of the most expensive part of the view generation.

The following diagram conceptualizes this in ZeroMQ:

zmq-processing

Still a work in progress...

Even if an ASP.NET application is architected and programmed for concurrency from the get-go, you are limited by the constraints of the hardware (# of concurrent threads).  Of course, you can add more servers and put a load balancer in front of them, but this can be an expensive proposition.  Perhaps a better architecture would be to design a system that allows adding cheap, heterogeneous server instances that do nothing but process parts of a view.

 In such an architecture, it would be possible to scale the system at any level by simply adding more nodes -- at any level.  They could be entirely heterogeneous; no need for IIS, in fact, the servers don't even have to be Windows servers.  The tradeoff is that you have to manage the session information yourself and push the relevant information down through the pipeline or at least make it accessible via a high speed interface (maybe like a Redis or Memcached?).

But the net gain is that it would allow for concurrent processing of a single web request and build an infrastructure for handling web requests that is easily scaled with cheap, simple nodes.

Filed under: .Net, Awesome, ZeroMQ No Comments
16Aug/12Off

SharePoint ListData.svc 500 Error

Posted by Charles Chen

If you're fighting with the SharePoint ListData.svc with an odd error:

An error occurred while processing this request.

And you are using an OData operator like endswith, you may encounter this error and be puzzled with why it works for some fields but not others.

Tried various theories -- indexed column?  use the column in a view?  maybe error with the column? -- with no love until Rob thought that it might have to do with empty values.

Turns out that the underlying implementation of ListData.svc doesn't quite like it if you have un-set or "null" values in your text fields.  So a query like this:

http://collab.dev.com/_vti_bin/ListData.svc/Test?$filter=endswith(PrimaryFaxNumber, '6481099') eq true

Will fail if there is an item in the list with an empty value for PrimaryFaxNumber.

However, using a nullity check will fix the issue:

http://collab.dev.com/_vti_bin/ListData.svc/Test?$filter=PrimaryFaxNumber ne null and endswith(PrimaryFaxNumber, '6481099') eq true
Filed under: .Net, SharePoint No Comments
15Sep/11Off

Now I REALLY Can’t be Bothered to Learn Silverlight

Posted by Charles Chen

I've blogged about it before, but seriously, the question has to be asked: if you're a developer with limited bandwidth to focus on mastering new technologies, why would you spend that time on Silverlight?

Not only is WP7 floundering, but now the news is out: the Metro version of IE 10 in Windows 8 won't support any plugins - including Silverlight:

Windows 8 will have two versions of Internet Explorer 10: a conventional browser that lives on the legacy desktop, and a new Metro-style, touch-friendly browser that lives in the Metro world. The second of these, the Metro browser, will not support any plugins. Whether Flash, Silverlight, or some custom business app, sites that need plugins will only be accessible in the non-touch, desktop-based browser.

Should one ever come across a page that needs a plugin, the Metro browser has a button to go to that page within the desktop browser. This yanks you out of the Metro experience and places you on the traditional desktop.

The rationale is a familiar one: plugin-based content shortens battery life, and comes with security, reliability, and privacy problems. Sites that currently depend on the capabilities provided by Flash or Silverlight should switch to HTML5.

If you're not on the HTML5 boat yet, I think the writing is on the wall: the Silverlight party is over (thank goodness).

Filed under: .Net, Awesome No Comments
13Sep/11Off

Lesson Learned on SharePoint Service Applications

Posted by Charles Chen

If you're setting out on writing your own SharePoint service applications, there is an important lesson that you should keep in mind (instead of learning it the hard way): ensure that all of your proxy, application proxy, service, service application, and service instance classes have public parameterless (default) constructors.

Otherwise, you'll have a heck of a time starting, instantiating, and uninstalling services with lots of MissingMethodExceptions and "{class} cannot be deserialized because it does not have a public default constructor" error messages.

Oddly enough, one thing I've learned from this is that the STSADM commands are often more "powerful" than the equivalent Powershell commands.  For example, Remove-SPSolution, even with the -force parameter, still failed with the aforementioned exceptions.  On the other hand, stsadm -o deletesolution {name} -override seemed to work fine.  Puzzling, for the moment, but it got the job done.  Similarly, stopping a service application that's AWOL (stuck on the processing screen) can be accomplished with stsadm -o provisionservice. Deleting it can be done using stsadm -o deleteconfigurationobject (though this one does seem to have side effects...).

Seems that Powershell is still a second class citizen when it comes to basic SharePoint command-line management.

But in any case, if you set out building your own service applications (<rant>and damn it Microsoft, can't you put some better examples out there?!  Even the few that are out there are convoluted, missing key details, hard to follow...</rant>), be sure to include public, default, parameterless constructors.

Filed under: .Net, Rants, SharePoint No Comments
29Jul/11Off

Working with GUIDs in MongoDB and ASP.NET MVC3

Posted by Charles Chen

Just a small tip for those looking to use GUIDs as document IDs in MongoDB in conjunction with ASP.NET MVC3: it's a lot more straightforward than it may seem at the onset.

These examples are based off of the ASP.NET MVC3 tutorials...except with MongoDB instead of EF+SQL Server.

I've set up my model class like so:

public class Movie
{
    [BsonId]
    public Guid ID { get; set; }
    public string Title { get; set; }
    public DateTime ReleaseDate { get; set; }
    public string Genre { get; set; }
    public decimal Price { get; set; }
}

When the application creates an object and persists it to the database, you'll see that it shows up like this in the Mongo console (I've formatted the JSON for clarity):

> db.movies.find()
{
   "_id":BinData(3,"n2FLBkAkhEOCkX42BGXRqg=="),
   "Title":"Test",
   "ReleaseDate":   ISODate("2011-05-11T04:00:00   Z"),
   "Genre":"Comedy",
   "Price":"9.99"
}

If you try to serialize this to JSON, instead of getting a GUID string, you'll get:

// Get a document
BsonDocument document = movies.FindOneAs<BsonDocument>();

// Direct to JSON
document.ToJson();
/*
{
   "_id":new BinData(3,
   "n2FLBkAkhEOCkX42BGXRqg=="   ),
   "Title":"Test",
   "ReleaseDate":   ISODate("2011-05-11T04:00:00   Z"),
   "Genre":"Comedy",
   "Price":"9.99"
}
*/

// With settings
JsonWriterSettings settings = new JsonWriterSettings{OutputMode = JsonOutputMode.JavaScript };
document.ToJson(settings);
/*
{
   "_id":{
      "$binary":"n2FLBkAkhEOCkX42BGXRqg==",
      "$type":"03"
   },
   "Title":"Test",
   "ReleaseDate":Date(1305086400000),
   "Genre":"Comedy",
   "Price":"9.99"
}
*/

This is somewhat inconvenient if you want to work with it from a pure JavaScript perspective; I was hoping that it would have returned a GUID as a string instead.  I was also concerned that this meant that I'd have to manage this manually as well on the server side in my actions, but it turns out that it works better than expected.  The only caveat is that you have to use "_id" when creating queries; otherwise, you can use the GUID as-is and the Mongo APIs will convert it behind the scenes:

public ActionResult Details(Guid id)
{
    MongoCollection<Movie> movies = _database.GetCollection<Movie>("movies");

    // No need to mess with the GUID; use it as is.
    Movie movie = movies.FindOneAs<Movie>(Query.EQ("_id", id)); 

    return View(movie);
}

You can see the result below in the browser:

Note the properly formatted GUID in the URL

So far, so good with my little Mongo+MVC3 experiment :D

Filed under: .Net, Mongo, MVC No Comments
5Jul/11Off

The Beauty of Mongo

Posted by Charles Chen

MongoDB is sexy.  Sexy as hell.

This isn't the first or the best intro to Mongo+.NET, but I hope this one can show you how it finally solves the object-relational disconnect with an easy to follow example.

Let's start with the model:

#region prologue

// Charles Chen

#endregion

using System;
using System.Collections.Generic;
using MongoDB.Bson;
using MongoDB.Bson.Serialization.Attributes;

namespace MongoAssessmentsTest
{
    public class Assessment
    {
        private DateTime _created;
        private string _description;
        [BsonId]
        private ObjectId _id;
        private List<ContentItem> _items;
        private DateTime _lastUpdated;
        private string _ownerId;
        private List<string> _tags;
        private string _title;

        public ObjectId Id
        {
            get { return _id; }
            set { _id = value; }
        }

        public string OwnerId
        {
            get { return _ownerId; }
            set { _ownerId = value; }
        }

        public string Title
        {
            get { return _title; }
            set { _title = value; }
        }

        public string Description
        {
            get { return _description; }
            set { _description = value; }
        }

        public List<string> Tags
        {
            get { return _tags; }
            set { _tags = value; }
        }

        public DateTime Created
        {
            get { return _created; }
            set { _created = value; }
        }

        public DateTime LastUpdated
        {
            get { return _lastUpdated; }
            set { _lastUpdated = value; }
        }

        public List<ContentItem> Items
        {
            get { return _items; }
            set { _items = value; }
        }
    }

    public class ContentItem
    {
        private List<Question> _questions;
        private string _text;

        public string Text
        {
            get { return _text; }
            set { _text = value; }
        }

        public List<Question> Questions
        {
            get { return _questions; }
            set { _questions = value; }
        }
    }

    [BsonKnownTypes(typeof(CheckboxQuestion), typeof(RadioQuestion), typeof(SelectQuestion), typeof(FreeTextQuestion))]
    public class Question
    {
        private int _order;
        private int _points;
        private string _text;

        public string Text
        {
            get { return _text; }
            set { _text = value; }
        }

        public int Order
        {
            get { return _order; }
            set { _order = value; }
        }

        public int Points
        {
            get { return _points; }
            set { _points = value; }
        }
    }

    public class CheckboxQuestion : Question
    {
        private List<string> _answers;
        private List<string> _choices;

        public List<string> Choices
        {
            get { return _choices; }
            set { _choices = value; }
        }

        public List<string> Answers
        {
            get { return _answers; }
            set { _answers = value; }
        }
    }

    public class SelectQuestion : Question
    {
        private string _answer;
        private List<string> _choices;

        public List<string> Choices
        {
            get { return _choices; }
            set { _choices = value; }
        }

        public string Answer
        {
            get { return _answer; }
            set { _answer = value; }
        }
    }

    public class RadioQuestion : SelectQuestion { }

    public class FreeTextQuestion : Question
    {
        private List<string> _acceptableAnswers;
        private string _isAnswerCaseSensitive;

        public string IsAnswerCaseSensitive
        {
            get { return _isAnswerCaseSensitive; }
            set { _isAnswerCaseSensitive = value; }
        }

        public List<string> AcceptableAnswers
        {
            get { return _acceptableAnswers; }
            set { _acceptableAnswers = value; }
        }
    }
}

The beauty of it is the absolute simplicity of working with this model.  Aside from a few attributes, there's not much thought that needs to be given to collections and inheritance hierarchies (though there are additional attributes and classes that can be used to control how these are stored if so desired).  I love it because this approach keeps your domain models clean.

How do we interact with this model?

#region prologue

// Charles Chen

#endregion

using System;
using System.Collections.Generic;
using MongoDB.Bson;
using MongoDB.Driver;
using MongoDB.Driver.Builders;

namespace MongoAssessmentsTest
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            Program program = new Program();
            program.Run();
        }

        private void Run()
        {
            MongoServer server = MongoServer.Create(); // Uses default connection options to localhost

            MongoDatabase database = server.GetDatabase("assessments_db");

            if (!database.CollectionExists("assessments"))
            {
                CommandResult result = database.CreateCollection("assessments");

                if (!result.Ok)
                {
                    Console.Out.WriteLine(result.ErrorMessage);
                    return;
                }
            }

            MongoCollection<Assessment> assessments = database.GetCollection<Assessment>("assessments");

            if (assessments.FindOne(Query.EQ("Title", "Sample 1")) == null)
            {
                // Insert an assessment.
                Assessment a = new Assessment
                {
                    Created = DateTime.Now,
                    Description = "A sample assessment",
                    Title = "Sample 1",
                    Items = new List<ContentItem>
                    {
                        new ContentItem
                        {
                            Questions = new List<Question>
                            {
                                new SelectQuestion
                                {
                                    Text = "Who was the first president of the United States?",
                                    Answer = "George Washington",
                                    Choices = new List<string>
                                    {
                                        "George Washington",
                                        "Abraham Lincoln",
                                        "John Adams"
                                    }
                                },
                                new CheckboxQuestion
                                {
                                    Text = "Which of these is NOT a former US President?",
                                    Answers = new List<string>
                                    {
                                        "Benjamin Franklin",
                                        "Hillary Clinton"
                                    },
                                    Choices = new List<string>
                                    {
                                        "Benjamin Franklin",
                                        "Bill Clinton",
                                        "Hillary Clinton",
                                        "Andrew Jackson",
                                        "James Garfield"
                                    }
                                }
                            }
                        }
                    }
                };

                assessments.Insert(a);
            }

            // Get it as BSON - great for writing straight to a web page
            BsonDocument a1 = assessments.FindOneAs<BsonDocument>(Query.EQ("Title", "Sample 1"));

            Console.Out.WriteLine(a1);

            // Get it as an object - great if you want to work with it on the server.
            Assessment a2 = assessments.FindOneAs<Assessment>(Query.EQ("Title", "Sample 1"));

            Console.Out.WriteLine(a2.Title);
        }
    }
}

Brilliantly simple.  Just fire up mongod.exe and you're ready to rock and roll.  Download the example from here: MongoAssessmentsTest.zip

Filed under: .Net, Dev, Mongo No Comments
22Jun/11Off

FluentNHibernate vs. Code First in EF 4.1

Posted by Charles Chen

First, having been in the SharePoint space exclusively for so long now, I have to admit: it's been a while since I've had to deal with non-SharePoint data access.  I don't know if I miss it or not :-D I've become really comfortable with my little SharePoint ORM approach thingy that generates models and such from content types (before you say SPMetal, this was developed for 2007 and I think still works better than SPMetal).

In the past, I've mostly avoided Entity Framework (EF), preferring LINQ-to-SQL due to EF1's shortcomings such as being less performant, creating more obtuse queries, my general disdain for the entity designer, the overly complex API, the lack of POCO support, etc.  I've also spent some time with NHibernate and FluentNHibernate and found it more pleasant to work with as an ORM solution.

Only recently have I discovered the "code first" approach released with EF4.1 which makes it much more appealing in the same way that FNH made NH more appealing by doing away with hbm.xml files for mapping your entities.  So I decided to take it for a spin and see how it measures up to NH+FNH.

If you're interested in much more in depth (and partisan) debate on the merits of one or the other, there's plenty to go around.  I won't get into that :-) I'm just concerned with the basics for now and I anticipate this being a series of blog posts as I test the merits -- and demerits -- of each.

For this first post, the basics are:

  • Create a simple console application that manages contacts
  • The application should auto-generate and auto-provision the schema
  • Basic queries should just work and not generate "dumb" SQL (i.e. counts should use COUNT in SQL, basic paging should be SQL based).

First up: Entity Framework.

So let's jump right into the code:

using System;
using System.Data.Entity;
using System.Linq;

namespace EFTest
{
    public class Program
    {
        private static void Main(string[] args)
        {
            Database.SetInitializer<ContactContext>(
                new DropCreateDatabaseAlways<ContactContext>());

            using (ContactContext context = new ContactContext())
            {
                // Basic add
                Contact chuck = new Contact {FirstName = "Charles", LastName = "Chen"};
                Contact sandra = new Contact {FirstName = "Sandra", LastName = "Chen"};
                Contact charlotte = new Contact {FirstName = "Charlotte", LastName = "Chen"};
                Contact judy = new Contact {FirstName = "Judy", LastName = "Chen"};

                context.Contacts.Add(chuck);
                context.Contacts.Add(sandra);
                context.Contacts.Add(charlotte);
                context.Contacts.Add(judy);

                context.SaveChanges();

                // Basic paged read
                var query = from c in context.Contacts
                               select c;

                var results = query.OrderBy(c => c.FirstName).Skip(2).Take(2);

                foreach(Contact c in results)
                {
                    Console.Out.WriteLine(c);
                }

                // Basic count
                int total = context.Contacts.Count();

                Console.Out.WriteLine(total);
            }
        }
    }

    public class Contact
    {
        public int ContactId { get; set; }
        public string FirstName { get; set; }
        public string LastName { get; set; }
        public override string ToString()
        {
            return string.Format("{0} {1}", FirstName, LastName);
        }
    }

    public class ContactContext : DbContext
    {
        public DbSet<Contact> Contacts { get; set; }
    }
}

I like that it's fairly compact and straightforward.  The development experience was a bit challenging, though.  First, EF doesn't like it when you try to use the schema with an existing database.  It insists that you let it provision the database.  Okay, fine (though there are workarounds).  It's often thrown around in these debates that one of the benefits of EF is that it's "out-of-the-box" but in reality, at least with the code first bits, it's anything but.  You have to download EF4.1 first and install it just like you would with NH+FNH (though certainly, that may change in the future).

The walkthrough on the ADO.NET team blog is also broken.  To get DbContext, you need to add a reference to EntityFramework.dll, not System.Data.Entity as posted in the blog.  Overall, I found the code to be more compact and easier to work with.  The one downside is the that one has to consistently update the class inheriting from DbContext as new entities are added.

Second up: NH+FNH:

Now lets take a look at code that does the same thing in NH:

using System;
using System.Collections.Generic;
using FluentNHibernate.Automapping;
using FluentNHibernate.Cfg;
using FluentNHibernate.Cfg.Db;
using NHibernate;
using NHibernate.Cfg;
using NHibernate.Tool.hbm2ddl;

namespace FNHTest
{
    internal class Program
    {
        private static void Main(string[] args)
        {
            ISessionFactory sessionFactory = CreateSessionFactory();

            using (ISession session = sessionFactory.OpenSession())
            using (ITransaction transaction = session.BeginTransaction())
            {
                // Basic add
                Contact chuck = new Contact {FirstName = "Charles", LastName = "Chen"};
                Contact sandra = new Contact {FirstName = "Sandra", LastName = "Chen"};
                Contact judy = new Contact {FirstName = "Judy", LastName = "Chen"};
                Contact charlotte = new Contact {FirstName = "Charlotte", LastName = "Chen"};

                session.SaveOrUpdate(chuck);
                session.SaveOrUpdate(sandra);
                session.SaveOrUpdate(judy);
                session.SaveOrUpdate(charlotte);

                transaction.Commit();

                // Basic paged read
                IList<Contact> results = session.QueryOver<Contact>()
                    .OrderBy(c => c.FirstName).Asc.Skip(2).Take(2).List();

                foreach (Contact c in results)
                {
                    Console.Out.WriteLine(c);
                }

                // Check the count
                int total = session.QueryOver<Contact>().RowCount();

                Console.Out.WriteLine(total);
            }
        }

        private static ISessionFactory CreateSessionFactory()
        {
            return Fluently.Configure()
                .Database(
                    MsSqlConfiguration.MsSql2008.ConnectionString("Data Source=CHARLES-E6410;Initial Catalog=FNHTest;Integrated Security=SSPI;Application Name='FNHTest'")
                )
                .Mappings(m => m.AutoMappings.Add(AutoMap.AssemblyOf<Program>()))
                .ExposeConfiguration(BuildSchema)
                .BuildSessionFactory();
        }

        private static void BuildSchema(Configuration config)
        {
            SchemaExport schema = new SchemaExport(config);

            schema.Drop(false, true); // Drops the tables only.
            schema.Create(false, true);
        }
    }

    public class Contact
    {
        public virtual int Id { get; set; }
        public virtual string FirstName { get; set; }
        public virtual string LastName { get; set; }

        public override string ToString()
        {
            return string.Format("{0} {1}", FirstName, LastName);
        }
    }
}

It's slightly more verbose, but not terribly so.  One note is that unlike the case with the ContactContext in EF, you won't need to continually update a "registry" with new entity types.

For this basic scenario, it's hard to say I prefer one over the other, but I'd have to give the edge to EF so far simply for the intangibles (read: Microsoft supported - in other words, it'll be easier from a convincing-your-team-to-not-roll-your-own standpoint).

Comparing the SQL

Of course, the next step is to compare the SQL generated from each of these.

Let's take a look at each query:

/*EF*/
SELECT TOP (2)
[Extent1].[ContactId] AS [ContactId],
[Extent1].[FirstName] AS [FirstName],
[Extent1].[LastName] AS [LastName]
FROM (
    SELECT
        [Extent1].[ContactId] AS [ContactId],
        [Extent1].[FirstName] AS [FirstName],
        [Extent1].[LastName] AS [LastName],
        row_number() OVER (ORDER BY [Extent1].[FirstName] ASC) AS [row_number]
    FROM [dbo].[Contacts] AS [Extent1]
)  AS [Extent1]
WHERE [Extent1].[row_number] > 2
ORDER BY [Extent1].[FirstName] ASC
/*NH*/
exec sp_executesql
N'SELECT TOP (@p0)
    Id0_0_,
    FirstName0_0_,
    LastName0_0_
FROM
(
    SELECT
        this_.Id as Id0_0_,
        this_.FirstName as FirstName0_0_,
        this_.LastName as LastName0_0_,
        ROW_NUMBER() OVER(ORDER BY this_.FirstName) as __hibernate_sort_row
    FROM
        [Contact] this_
) as query
WHERE
    query.__hibernate_sort_row > @p1 ORDER BY query.__hibernate_sort_row',N'@p0 int,@p1 int',@p0=2,@p1=2

You can see that for the first, paged read, EF uses a straight SQL statement whereas NH uses a parameterized dynamic SQL statement.  For this small dataset, there is no discernible difference in performance, but I would gather that for larger datasets, we'd see a performance boost with NH.

For the second query to count, we see that again, there is a small difference in how the two go about generating queries:

/*EF*/
SELECT
[GroupBy1].[A1] AS [C1]
FROM ( SELECT
	COUNT(1) AS [A1]
	FROM [dbo].[Contacts] AS [Extent1]
)  AS [GroupBy1]
/*FNH*/
SELECT count(*) as y0_ FROM [Contact] this_

As far as I can tell, for this basic scenario where there is no additional filtering on columns, there should be no practical performance difference between the two (though obviously, EF generates an extra nested select statement, the real world performance impact is negligible).

Next up: I want to do some more tests with more complex queries and figure out how each of these frameworks handles dealing with schema changes.  Seems like a wash for these basic scenarios and one of personal preference.

Filed under: .Net, Dev, SQL Server 2 Comments
4Mar/11Off

Getting Version Changes From SharePoint

Posted by Charles Chen

SharePoint doesn't provide a straightforward way to do this it seems.  My first crack was to try to leverage the BeforeProperties and AfterProperties in an event receiver, but it turns out that this doesn't work quite as you would expect based on the naming of these two properties (and the same goes for ChangedProperties).

However, clearly we can see that the version history contains the changes in the fields from one version to the next.

Here's some basic code to identify the fields that changed from one version to the next:

using (SPSite site = new SPSite("http://my.domain.com/"))
using (SPWeb web = site.OpenWeb())
{
    SPList list = web.GetList("http://my.domain.com/Lists/Stuff");
    SPListItem item = list.Items[1];

    Console.Out.WriteLine(item.Title);

    SPListItemVersion currentVersion = item.Versions[0];
    SPListItemVersion previousVersion = item.Versions.Count > 1 ? item.Versions[1] : null;

    Console.Out.WriteLine("Version is current: {0}", currentVersion.IsCurrentVersion);

    foreach (SPField field in currentVersion.Fields)
    {
        if (field.ShowInVersionHistory == false)
        {
            continue;
        }

        if (previousVersion == null)
        {
            Console.Out.WriteLine("  > {0} changed to \"{1}\"",
                field.StaticName, currentVersion[field.StaticName]);
            continue;
        }

        if (currentVersion[field.StaticName].Equals(previousVersion[field.StaticName]))
        {
            continue;
        }

        Console.Out.WriteLine("  > {0} changed from \"{1}\" to \"{2}\"",
            field.StaticName, previousVersion[field.StaticName], currentVersion[field.StaticName]);
    }
}

This will get you the changes between the current version and the previous version.  If it's a new object, all of the changes will be returned.

Filed under: .Net, SharePoint No Comments
18Feb/11Off

Most Annoying Thing About SharePoint 2010?

Posted by Charles Chen

I've been bashing my head against a SharePoint 2010 solution package for almost 10 hours now across two days.  I mean god forbid Microsoft provides us poor developers with some useful error messages or even a hint of what's going on inside that crazy contraption.

It seems that other's have also encountered this problem, but I'll summarize here: in SharePoint 2007, when you create a custom list template and list definition, you could associated a custom content type with the list quite easily (or so I thought...more on this in a sec).  Here's an example from my schema.xml file:

<?xml version="1.0" encoding="utf-8"?>
<List xmlns:ows="Microsoft SharePoint"
	Title="Basic List" FolderCreation="FALSE" Direction="$Resources:Direction;"
	Url="Lists/Basic List" BaseType="0" EnableContentTypes="TRUE">
	<MetaData>
		<ContentTypes>
			<ContentTypeRef ID="0x0100C171C000000000000000000000000003"/>
			<ContentTypeRef ID="0x0120" />
		</ContentTypes>

<!-- additional markup omitted -->

I can then use the content type fields in the views without having add the fields or field references separately:

<ViewFields>
	<FieldRef Name="LinkTitleNoMenu"></FieldRef>
	<FieldRef Name="Request_Name" ></FieldRef>
	<FieldRef Name="Request_Status" ></FieldRef>
	<FieldRef Name="Requestor" ></FieldRef>
	<FieldRef Name="Ticket_Project_Number" ></FieldRef>
	<FieldRef Name="Ticket_Group" ></FieldRef>
	<FieldRef Name="Support_Ticket_Number" ></FieldRef>
	<FieldRef Name="Support_Ticket_Name" ></FieldRef>
	<FieldRef Name="Support_Ticket_Status" ></FieldRef>
	<FieldRef Name="Support_Ticket_Res_Date" ></FieldRef>
	<FieldRef Name="Technical_Manager" ></FieldRef>
</ViewFields>

In SharePoint 2010, this doesn't work. Let me rephrase this: it doesn't work 100% correctly.  The default behavior seems to be that the content type will be copied over, but the fields aren't copied over...

Well in fact, Microsoft's documentation states that this is as designed:

When SharePoint Foundation creates a list instance, it includes only those columns that are declared in the base type schema of the list or in the list schema. If you reference a site content type in the list schema, and that content type references site columns that are not included in the base type schema of the list or in the list schema, those columns are not included. You must declare those columns in the list schema for SharePoint Foundation to include them on the list.

What you'll get is the content type will indeed be on the list, but the fields won't be copied over and any views that use those fields will be borked.  This is a really annoying problem.  Really, it is.

The solution is to either build some tool that does an XSL transform on my content types and produces the views (thereby avoiding the necessity of copying the fields over manually) or write some code to fix it after the list has deployed.  But it's problematic because I want to deploy data with my list instance using markup like so:

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
    <ListInstance
		Title="My Custom Configuration"
		Id="18000"
		FeatureId="ACA62A34-7B27-442D-97C6-55FC3F55A2BF"
		TemplateType="18000"
		Url="Lists/MyCustomConfiguration"
        OnQuickLaunch="TRUE">
      <Data>
          <Rows>
              <Row>
                  <Field Name="ID">1</Field>
                  <Field Name="Title">some.setting.name</Field>
                  <Field Name="Configuration_Value">http://myserver.mydomain.com</Field>
                  <Field Name="Default_Value">http://myserver.mydomain.com</Field>
                  <Field Name="Usage_Description">The URL of the something something server</Field>
                  <Field Name="Property_Name">SomethingSomethingServer</Field>
                  <Field Name="Property_Type">string</Field>
              </Row>
          </Rows>
      </Data>
    </ListInstance>
</Elements>

I need to be able to deploy the content type to the list automatically.  (If you're not deploying default data with your list, the ContentTypeBinding element may do the trick.)

So I fumbled around with this for a day and a half while bitching and moaning about it to anyone who would listen to me.  Did I screw up the content type somehow?  Did I screw up the list template somehow?  Did I mess up an ID somewhere?  I mean, I've done this a million times in the past in a million different solution packages and even in SharePoint 2010...what the heck was different about this package?

Finally (in fact, just moments ago!) it dawned on me that I've always "primed" my content types when deploying them in my previous solutions based on some findings from Martin Hatch!

You do not have to declare custom fields in the schema.xml, as long as you manually update your content type before creating any libraries.

Bingo!

Martin links to a code sample, but the gist of it is pretty simple.

  1. Wire up a feature receiver for your content type feature.
  2. On feature activation, "prime" the content types that have a specific group name
  3. ???
  4. Profit!

Okay, the profiting part is questionable, the the rest of it isn't!

The first part is to wire up the feature for a feature receiver.

<Feature Id="0C4C1210-B056-4AFC-B556-A8BB30E1A9F1"
         Title="My Custom Content Types"
         Description="Installs the content types."
         Hidden="FALSE"
         Scope="Site"
         DefaultResourceFile="core"
         ReceiverAssembly="MyAssembly, Version=1.0.0.0, Culture=neutral, PublicKeyToken=56f39f66bb4e1d17"
         ReceiverClass="MyAssembly.Components.FeatureReceivers.MetadataPrimer"
         xmlns="http://schemas.microsoft.com/sharepoint/">
    <ElementManifests>
        <ElementManifest Location="my_contenttypes.xml" />
    </ElementManifests>
    <Properties>
        <Property Key="prime.if.contains" Value="My Custom Content Type"/>
    </Properties>
</Feature>

Note that I've added a feature property here: a key called "prime.if.contains".  This allows me to specify the group names of the content types that I want to prime.  That way, I don't have to prime all of the content types.

I create a base class for my feature receivers:

/// <summary>
///     Abstract base class for custom feature receivers.
/// </summary>
/// <remarks>
///     All custom feature receivers should be updated to inherit from this class for common functionality.
/// </remarks>
public abstract class FeatureReceiverBase : SPFeatureReceiver
{
	/// <summary>
	///     Gets the <c>SPWeb</c>.
	/// </summary>
	/// <remarks>
	///     This method allows the feature receiver to obtain a reference to the current <c>SPWeb</c>
	///     regardless of whether there is an <c>HttpContext</c> (i.e. in the case that the feature
	///     is activated from <c>STSADM</c>).
	/// </remarks>
	/// <param name = "properties">The properties.</param>
	/// <returns>The <c>SPWeb</c> instance from the properties.</returns>
	protected static SPWeb GetWeb(SPFeatureReceiverProperties properties)
	{
		SPWeb web;

		if (SPContext.Current != null && SPContext.Current.Web != null)
		{
			web = SPContext.Current.Web;
		}
		else if (properties.Feature.Parent is SPWeb)
		{
			web = (SPWeb) properties.Feature.Parent;
		}
		else if (properties.Feature.Parent is SPSite)
		{
			web = ((SPSite) properties.Feature.Parent).RootWeb;
		}
		else
		{
			throw new Exception("Unable to retrieve SPWeb - this feature is not site or web-scoped.");
		}

		return web;
	}
}

And then I add the primer class:

/// <summary>
///     <para>Receiver which "primes" the content types by updating each of the custom content types to push the changes down.</para>
///     <para><see cref = "http://mkeeper.spaces.live.com/blog/cns!60F12A60288E5607!278.entry" /></para>
/// </summary>
public class MetadataPrimer : FeatureReceiverBase
{
	private const string _primerKey = "prime.if.contains";

	/// <summary>
	///     Handles the feature activated event by priming content types specified by the primer key property.
	/// </summary>
	/// <param name = "properties">The properties.</param>
	public override void FeatureActivated(SPFeatureReceiverProperties properties)
	{
		Console.Out.WriteLine("Activating Metadata Primer");

		// Prime the content types.
		SPWeb web = GetWeb(properties);

		SPContentTypeCollection contentTypes = web.ContentTypes;

		List<string> contentTypeIds = new List<string>();

		// Capture the feature properties.
		Dictionary<string, string> featureProperties = new Dictionary<string, string>();

		foreach (SPFeatureProperty property in properties.Feature.Properties)
		{
			featureProperties[property.Name] = property.Value;
		}

		// Must find the key "prime.if.contains"
		if (!featureProperties.ContainsKey(_primerKey))
		{
			ExceptionFactory.Throw<ArgumentException>(
				"The property prime.if.contains is not defined in the feature.");
		}

		string[] tokens = featureProperties[_primerKey].Split(new[] {','}, StringSplitOptions.RemoveEmptyEntries);

		Log("Priming content types containing: {0}", featureProperties[_primerKey]);

		foreach (SPContentType contentType in contentTypes)
		{
			bool prime = tokens.Any(token => contentType.Group.ToUpperInvariant().Contains(token.ToUpperInvariant()));

			if (!prime)
			{
				continue; // EXIT: Next iteration.
			}

			contentTypeIds.Add(contentType.Id.ToString());

			Log("Adding content type \"{0}\" ({1}) for priming", contentType.Name, contentType.Id);
		}

		// Do it in two loops since doing it one causes issues if a parent content type is primed (child content types
		// fail to prime because the parent changes).
		foreach (string id in contentTypeIds)
		{
			try
			{
				SPContentTypeId contentTypeId = new SPContentTypeId(id);

				SPContentType contentType = web.ContentTypes[contentTypeId];

				Log("Priming content type: {0}", contentType.Name);

				contentType.Update(true, false);
			}
			catch (Exception)
			{
				Log("Failed to prime content type with ID: {0}", id);
			}
		}
	}

	/// <summary>
	///     Logs the specified message to both log4net and the console.
	/// </summary>
	/// <param name = "message">The message.</param>
	/// <param name = "args">The args.</param>
	private static void Log(string message, params object[] args)
	{
		Console.Out.WriteLine(message, args);
	}

	/// <summary>
	///     Features the deactivating.
	/// </summary>
	/// <param name = "properties">The properties.</param>
	public override void FeatureDeactivating(SPFeatureReceiverProperties properties) {}

	/// <summary>
	///     Features the installed.
	/// </summary>
	/// <param name = "properties">The properties.</param>
	public override void FeatureInstalled(SPFeatureReceiverProperties properties) {}

	/// <summary>
	///     Features the uninstalling.
	/// </summary>
	/// <param name = "properties">The properties.</param>
	public override void FeatureUninstalling(SPFeatureReceiverProperties properties) {}
}

(Note: remove or replace the usages of ExceptionFactory -- this is just a custom exception helper that I use)

You can see, the gist of it is that the code will iterate all of the content types at the site, find ones that belong in the given groups, and them prime the content type by calling SPContentType.Update(bool, bool).

So there you have it; it didn't work in 2007 either, but I had completely forgotten that I had written the primer for this express purpose.  With this simple bit of code, you can save yourself the hassle of having to copy fields from your content type to your schema.xml definition file when creating custom list definitions for SharePoint and all of the fields are nicely copied over from your site level content type to your list instance without having to manually modify the schema.xml file for your list.

Filed under: .Net, SharePoint Comments Off