<CharlieDigital/> Programming, Politics, and uhh…pineapples


Can I Hadoop My Way Out of This?

Posted by Charles Chen

Quite possibly one of the funniest slides I've ever seen:

Courtesy of Jared Rosoff

It's from a slidedeck on real-time analytics and MongoDB.

I guess one cannot simply Hadoop their way out of that type of mess.


How NOT to Implement a Paywall

Posted by Charles Chen

Yeah...I hope they didn't pay these guys too much.

Via: Cynical-C.

Filed under: lulz, Miscellany No Comments

FluentNHibernate vs. Code First in EF 4.1

Posted by Charles Chen

First, having been in the SharePoint space exclusively for so long now, I have to admit: it's been a while since I've had to deal with non-SharePoint data access.  I don't know if I miss it or not 😀 I've become really comfortable with my little SharePoint ORM approach thingy that generates models and such from content types (before you say SPMetal, this was developed for 2007 and I think still works better than SPMetal).

In the past, I've mostly avoided Entity Framework (EF), preferring LINQ-to-SQL due to EF1's shortcomings such as being less performant, creating more obtuse queries, my general disdain for the entity designer, the overly complex API, the lack of POCO support, etc.  I've also spent some time with NHibernate and FluentNHibernate and found it more pleasant to work with as an ORM solution.

Only recently have I discovered the "code first" approach released with EF4.1 which makes it much more appealing in the same way that FNH made NH more appealing by doing away with hbm.xml files for mapping your entities.  So I decided to take it for a spin and see how it measures up to NH+FNH.

If you're interested in much more in depth (and partisan) debate on the merits of one or the other, there's plenty to go around.  I won't get into that 🙂 I'm just concerned with the basics for now and I anticipate this being a series of blog posts as I test the merits -- and demerits -- of each.

For this first post, the basics are:

  • Create a simple console application that manages contacts
  • The application should auto-generate and auto-provision the schema
  • Basic queries should just work and not generate "dumb" SQL (i.e. counts should use COUNT in SQL, basic paging should be SQL based).

First up: Entity Framework.

So let's jump right into the code:

using System;
using System.Data.Entity;
using System.Linq;

namespace EFTest
    public class Program
        private static void Main(string[] args)
                new DropCreateDatabaseAlways<ContactContext>());

            using (ContactContext context = new ContactContext())
                // Basic add
                Contact chuck = new Contact {FirstName = "Charles", LastName = "Chen"};
                Contact sandra = new Contact {FirstName = "Sandra", LastName = "Chen"};
                Contact charlotte = new Contact {FirstName = "Charlotte", LastName = "Chen"};
                Contact judy = new Contact {FirstName = "Judy", LastName = "Chen"};



                // Basic paged read
                var query = from c in context.Contacts
                               select c;

                var results = query.OrderBy(c => c.FirstName).Skip(2).Take(2);

                foreach(Contact c in results)

                // Basic count
                int total = context.Contacts.Count();


    public class Contact
        public int ContactId { get; set; }
        public string FirstName { get; set; }
        public string LastName { get; set; }
        public override string ToString()
            return string.Format("{0} {1}", FirstName, LastName);

    public class ContactContext : DbContext
        public DbSet<Contact> Contacts { get; set; }

I like that it's fairly compact and straightforward.  The development experience was a bit challenging, though.  First, EF doesn't like it when you try to use the schema with an existing database.  It insists that you let it provision the database.  Okay, fine (though there are workarounds).  It's often thrown around in these debates that one of the benefits of EF is that it's "out-of-the-box" but in reality, at least with the code first bits, it's anything but.  You have to download EF4.1 first and install it just like you would with NH+FNH (though certainly, that may change in the future).

The walkthrough on the ADO.NET team blog is also broken.  To get DbContext, you need to add a reference to EntityFramework.dll, not System.Data.Entity as posted in the blog.  Overall, I found the code to be more compact and easier to work with.  The one downside is the that one has to consistently update the class inheriting from DbContext as new entities are added.

Second up: NH+FNH:

Now lets take a look at code that does the same thing in NH:

using System;
using System.Collections.Generic;
using FluentNHibernate.Automapping;
using FluentNHibernate.Cfg;
using FluentNHibernate.Cfg.Db;
using NHibernate;
using NHibernate.Cfg;
using NHibernate.Tool.hbm2ddl;

namespace FNHTest
    internal class Program
        private static void Main(string[] args)
            ISessionFactory sessionFactory = CreateSessionFactory();

            using (ISession session = sessionFactory.OpenSession())
            using (ITransaction transaction = session.BeginTransaction())
                // Basic add
                Contact chuck = new Contact {FirstName = "Charles", LastName = "Chen"};
                Contact sandra = new Contact {FirstName = "Sandra", LastName = "Chen"};
                Contact judy = new Contact {FirstName = "Judy", LastName = "Chen"};
                Contact charlotte = new Contact {FirstName = "Charlotte", LastName = "Chen"};



                // Basic paged read
                IList<Contact> results = session.QueryOver<Contact>()
                    .OrderBy(c => c.FirstName).Asc.Skip(2).Take(2).List();

                foreach (Contact c in results)

                // Check the count
                int total = session.QueryOver<Contact>().RowCount();


        private static ISessionFactory CreateSessionFactory()
            return Fluently.Configure()
                    MsSqlConfiguration.MsSql2008.ConnectionString("Data Source=CHARLES-E6410;Initial Catalog=FNHTest;Integrated Security=SSPI;Application Name='FNHTest'")
                .Mappings(m => m.AutoMappings.Add(AutoMap.AssemblyOf<Program>()))

        private static void BuildSchema(Configuration config)
            SchemaExport schema = new SchemaExport(config);

            schema.Drop(false, true); // Drops the tables only.
            schema.Create(false, true);

    public class Contact
        public virtual int Id { get; set; }
        public virtual string FirstName { get; set; }
        public virtual string LastName { get; set; }

        public override string ToString()
            return string.Format("{0} {1}", FirstName, LastName);

It's slightly more verbose, but not terribly so.  One note is that unlike the case with the ContactContext in EF, you won't need to continually update a "registry" with new entity types.

For this basic scenario, it's hard to say I prefer one over the other, but I'd have to give the edge to EF so far simply for the intangibles (read: Microsoft supported - in other words, it'll be easier from a convincing-your-team-to-not-roll-your-own standpoint).

Comparing the SQL

Of course, the next step is to compare the SQL generated from each of these.

Let's take a look at each query:

[Extent1].[ContactId] AS [ContactId],
[Extent1].[FirstName] AS [FirstName],
[Extent1].[LastName] AS [LastName]
        [Extent1].[ContactId] AS [ContactId],
        [Extent1].[FirstName] AS [FirstName],
        [Extent1].[LastName] AS [LastName],
        row_number() OVER (ORDER BY [Extent1].[FirstName] ASC) AS [row_number]
    FROM [dbo].[Contacts] AS [Extent1]
)  AS [Extent1]
WHERE [Extent1].[row_number] > 2
ORDER BY [Extent1].[FirstName] ASC
exec sp_executesql
        this_.Id as Id0_0_,
        this_.FirstName as FirstName0_0_,
        this_.LastName as LastName0_0_,
        ROW_NUMBER() OVER(ORDER BY this_.FirstName) as __hibernate_sort_row
        [Contact] this_
) as query
    query.__hibernate_sort_row > @p1 ORDER BY query.__hibernate_sort_row',N'@p0 int,@p1 int',@p0=2,@p1=2

You can see that for the first, paged read, EF uses a straight SQL statement whereas NH uses a parameterized dynamic SQL statement.  For this small dataset, there is no discernible difference in performance, but I would gather that for larger datasets, we'd see a performance boost with NH.

For the second query to count, we see that again, there is a small difference in how the two go about generating queries:

[GroupBy1].[A1] AS [C1]
	COUNT(1) AS [A1]
	FROM [dbo].[Contacts] AS [Extent1]
)  AS [GroupBy1]
SELECT count(*) as y0_ FROM [Contact] this_

As far as I can tell, for this basic scenario where there is no additional filtering on columns, there should be no practical performance difference between the two (though obviously, EF generates an extra nested select statement, the real world performance impact is negligible).

Next up: I want to do some more tests with more complex queries and figure out how each of these frameworks handles dealing with schema changes.  Seems like a wash for these basic scenarios and one of personal preference.

Filed under: .Net, Dev, SQL Server 2 Comments

Lev Grossman on The Cloud

Posted by Charles Chen

Lev Grossman has an article in Time this week (not available on Time.com) that pretty much hits it on the mark -- at least in my book -- with regards to The Cloud.  I look kind of befuddled when people tell me we should do this or that on The Cloud or how their product is a Cloud solution.

Grossman lays into this in his opening paragraphs:

The best thing about cloud computing is that word: cloud. Telling consumers that their data is in the cloud is like telling a kid his dog has gone to doggie heaven.  There is no doggie heaven, and your data isn't in a cloud.  It's in a windowless, fortress-like data center somewhere in the rural U.S.

Cloud computing is just a buzzword companies use to describe what they're doing when they move data and processing tasks you're used to hosting on your personal computer onto their servers, which you can access via the Internet.  It isn't new; far from it.  It's at least as old as webmail services like Hotmail.  It just didn't have a cool name back then.

Of course, The Cloud has its merits and convenience (for consumer applications) is surely one of those merits as is scalability (for enterprises and businesses); however -- as Grossman argues -- one of the biggest pitfalls of The Cloud is the lack of control over you data.  Grossman continues:

But in some ways, the cloud is a step backward.  It harks back to computing's primordial past, when everything was cloud computing -- dumb terminals connected to central mainframes.

The thing is, I'm not sure I want my computer to be just a device.  Cloud computing goes hand in hand with another trend: the netbookization and iPadization of the PC, with its transformation into a beautifully designed but lobotomized device that relies on an Internet umbilical cord to do most of its actual computing.

As for me, from a development perspective I'm not too caught up in The Cloud hype.  For most purposes, unless you really know that you have a hit on your hands, you can host your applications much, much cheaper on shared hosting for about $10/mo. which is still probably the best way for a small business to get started.  And when you need to scale, well, hopefully, you'll have tons of investment capital at that point, too and you can just port your app to The Cloud.


Where Pakistan Went Wrong

Posted by Charles Chen

Okay, so there's lots of things wrong with Pakistan these days, but there was an interesting bit in an editorial in Newsweek by A.Q. Khan -- Pakistan's notorious nuclear architect:

On Dec. 10, 1984, I informed Gen. Zia-ul-Haq that we could explode a device at a week’s notice, whenever he so desired. We achieved credible nuclear capacity by the second half of the ’80s, and the delivery system was perfected in the early ’90s. For a country that couldn’t produce bicycle chains to have become a nuclear and missile power within a short span—and in the teeth of Western opposition—was quite a feat.

I have to wonder: how would Pakistan have turned out if, instead of focusing its resources on building up its nuclear program, it invested in the factories, the foundries, the workers, the mining operations, the transportation networks, and so on for manufacturing, distributing, and selling bicycle chains (for example) instead.  I think the world -- and Pakistan -- would have been a better place for it.

Perhaps they would have turned out more like South Korea or Taiwan instead of remaining a dangerous, unstable, largely third-world country with little prospect for economic and industrial progress.

It's sad that a guy who could figure out the intricacies and complexities of constructing a nuclear weapon has lost all perspective on what really would have advanced Pakistan and would have been a far more worthy investment of Pakistan's resources: creating gainful employment opportunities, building industry and commerce, and creating hope for their people.


FFMPEG and Flowplayer

Posted by Charles Chen

Flowplayer is a great little bit of Flash and JavaScript for incorporating streaming video on your site (until we get a fully viable, widely supported HTML5 solution).

I've used it in various capacities for putting together apps to stream recorded webcasts using CamStudio and splicing video and audio clips together in Windows Live Movie Maker (plenty adequate for my simple usage).  You can see an example of this on the home page for zaanglabs.

The input video was a 720p .wmv file output from Movie Maker that comes in at 13.7MB.  The output is a 3MB h.264 encoded .flv that is almost as good as the original.

For anyone doing webcasts using open source tools (or you just need to encode whatever you output from Movie Maker), this is what you'll need to pull it all together:

FFMPEG -i input.wmv -ar 44100 -qscale 9 -vcodec libx264 output.flv

The input is the file saved from Movie Maker and the output is a file that can be streamed using Flowplayer.  That seems to be the secret sauce after playing around with various settings and different configuration options.  This yields an output video that should be nearly indistinguishable from the original for webcasting purposes.  I'm sure you'll have to adjust the value if you have more action in your frames.

Happy webcasting!


More Commentary on Windows 8 and HTML5

Posted by Charles Chen

Arstechnica has another feature on the aftermath of the Microsoft's game-changing Windows 8 + HTML5 announcement.

First, a quote:

The lack of broad platform support meant that Silverlight could never quite rival Flash on this front, but it was there, and it worked well on those platforms that were supported. With Internet Explorer 9, however, Silverlight took a back seat. HTML5 became the way forward. If Silverlight were to be used at all, it should only be used for those things that HTML5 couldn't do very well, such as streaming video. For anything else, the message was that developers should use HTML5.

Microsoft did have a point. If you're really wanting to target people on any platform, HTML5 is the way to go. For Web-facing applications that don't have any special needs such as DRM video, HTML5 is the long-term bet.

Excuse the smugness, but I've been saying this from the day Silverlight was released (you know, because it's basically Flash and we all know how everyone loves the Flash website experience...).  It's plain silly to suggest using Silverlight for anything other than an intranet environment where DHTML could do the job.

While Peter Bright opens the article with a discussion of the teeth-gnashing that many Windows platform developers are going through, I -- for one -- could not be happier that we'll finally see HTML5 on the desktop.  I think it's a beautiful thing  Finally: the richness and variety that we've seen in web based applications will manifest on the desktop because the skills and techniques that folks have used for over a decade to build beautiful web applications will now finally translate.

However, I do disagree with a few points that Bright makes:

Where Silverlight programs can deal with buttons, icons, list boxes, tree views, and other interface controls, HTML5 applications must generally deal with boxes of text, with no higher-level concepts to work with.

There are a few points to address here.  The first is that -- obviously -- HTML has buttons and icons (<a class='icon'><img /></a>)  😉  The second is that, to a large degree, deficiencies in the area of higher level UI abstractions in the DHTML world have largely been handled by any number of open-source UI script and widget libraries which provide even better alternatives to those on the Windows native platform.  jsTree is one example of an excellent tree view library that is incredibly rich.  Third, having worked with tree views via DOM and tree views via WinForms and WPF, my personal feeling is that it is many, many times easier to deal with the creation, manipulation, and styling of HTML than it is with Windows forms controls.  And finally, I believe that Bright misses the point of HTML+CSS; it's really not much different from the concepts put forth in WPF and Silverlight: that of applying visual styles to otherwise generic controls.

I also take issue with this assertion from Bright:

Another weak area for HTML5 is tooling. Design and development tools that work with HTML5 are not as developed or as robust as those that exist for Silverlight, making HTML5 development more complicated

Uh....HELLO?!?  The term "HTML5" is a bit misleading as it's really nothing more than HTML4 with some new tags (and some new skills required to take advantage of those new tags).  Why are current HTML tools insufficient?  I'm not getting his point here as HTML development tools have been around for WAAAAAY longer than Silverlight or WPF development tools.

Bright closes the article with more doom-and-gloom and a bit of hope for going back to the good old days of the same-old, same-old:

The prospect of being stuck with HTML5 and JavaScript for their development is encouraging them to jump ship.

The details aren't clear yet, but next time we'll take a look at the pieces of the puzzle we have, and we'll learn why Windows 8 won't be a HTML-driven horror after all.

Apparently the reaction of Windows platform developers to the news that Windows 8 will promote HTML5 and JS as one alternative to building applications for the desktop

Contrary to Bright's doom-and-gloom, I'm fairly optimistic and I really, really, really hope that Microsoft sticks to its guns here.  I can't be the only person who, while working with WPF, longed dearly for a jQuery-esque framework to traverse and manipulate XAML the same way that jQuery enables in web applications.  I can't be the only person who's turned to using the WebBrowser control in Windows applications from time-to-time because it offered the simplest way of handling complex document layouts and, in some cases, interfaces that would require developers to resort to low level drawing of rectangles and custom painting of the UI controls and dozens to hundreds of lines of code that could be accomplished by HTML in a few lines of markup.

As a developer who has worked with DHTML, WinForms, and WPF, I -- for one -- look forward to an HTML5-on-the-desktop future.

PS  — Is it just me, or are MS developers -- generally speaking -- a particularly whiny bunch that can't handle radical shifts in technology or methodology very well because Microsoft and Visual Studio has been holding their hands and enabling their ineptitude for so long?  The comments on that Arstechnica article shows the straight FEAR that this news has instilled in the hearts of those legion developers.  I love it.

A couple of commenters hit it right on the mark:

slogger: If you have a CS degree and you can't functionally use a new language within a couple months, you fail and are worthless. Switching to preferring some sort of JS API is hardly the end of the world.

jaman4dbz: First of all: Man up MS developers. This is why you should pay attention when you get your formal education. If you did you can switch languages easily. The language and libraries should not matter much.

Maybe now all those retarded MS developers can learn some theory, rather than what buttons to hit in their program maker.

deet: Nobody's talking about using HTML to write drivers, for fucksake.

emn13: I've written apps in Silverlight, WPF, and HTML+javascript. They obviously have their own pro's and con's, but this articles' assertion that Silverlight/WPF are more high-level is complete nonsense.

One of the huge problems of WPF/Silverlight is that they're very *low-level*. WPF doesn't have a sane styling system. In fact, styles are implemented as effectively side-effect laden getters+setters. This means it's quite hard to separate the content and behavior from the presentation.

WPF comes with a widely-used grid-based layout scheme (as opposed to HTML's tables). This is nice in some ways since it's easy to precisely specify where something goes on that grid (i.e. it's low-level). It's a terrible pain to use by hand however, since grid-alignment is done by row/column *number*; moving things about generally means renumbering everything.

WPF/Silverlight as general UI-specification language, I'd say: good riddance. And as a special-purpose tool to make particularly intricate, tightly controlled UI's? Well, it's not disappearing, is it; if you want more control but with helpful automatic layout system to do the basics, WPF/Silverlight will still be around.

This comment by bouncing deserves its own shoutout:

Hmm. Peter Bright makes an interesting case.

In fact, a crack team of usability experts, computer science PhD candidates, and Seth Godin impersonators recently compiled an exhaustive list of native apps which are, to any reasonable observer, far superior to web apps:

  • Photoshop
  • Various CAD programs

Although only two native Windows apps that do not suck were ever found, researchers believe the third "non-sucky .exe file" may one day be discovered. Unfortunately, locating the mythical trifectapp has proven a daunting task, as finding it requires digging through tens of thousands of incredibly crappy, hard to use, slow, confusing, mind-numbing, rage-inducing, serial-killer-inspiring "line-of-business" apps that IT managers force their interns to use on their behalf after booting a USB thumb drive flashed with a version of Windows 98 that's compatible with the version of Microsoft's far-superior C++ libraries the IT department develops on.

Perhaps Peter Bright, who wrote this passionate defense of the unnamed third native application which does not suck will tell us as to what, if any, business application is better in native Win32 than HTML 1.0. While he's doing so, a 22 year-old Ruby programmer will create a superior product using "primitive" technologies in exactly 1/100th of the time it takes a team of Certified Win32 Development Professionals to follow a stack trace in Visual Studio.

I'd also like to point out one of the stupidest comments that simply perpetuates this myth that JavaScript is inadequate:

VoodooTrucker: Is JS as mature as C#? No way. In fact its totally painful, which is what I think developers are afraid of. No strong-typing, no obfuscation, no refactoring, no generics, no direct casts, no direct disk access, minimal properties, no LINQ, no IEnumerable, no interfaces, crappy inheritance, crappy dictionaries....

This kind of makes me chuckle a bit as my own perspective has been that as C# has developed from 1.0 to its current iteration, it has taken more and more from JavaScript.  Take collection and dictionary initializers in C#.  Looks familiar?  Because it was already done in JavaScript ages ago.  Trucker's assertions are also completely off.

It's true that because JavaScript is interpreted, there's no obfuscation, but there is typing in JavaScript, you can clearly refactor in JavaScript, you don't need generics in JavaScript because an array can hold anything, you can perform direct casts by creating a new instance of a type (i.e. var five = new Number("5")), no direct disk access is a matter of the platform/runtime rather than the language itself, no IEnumerable is a misnomer because really, you can iterate over anything in JavaScript, crappy dictionaries make me laugh because JavaScript dictionaries are incredibly sexy and flexible.

Filed under: Technology No Comments

Web Based Chat for SharePoint – ChatPoint

Posted by Charles Chen

Creative name, right 🙂

In any case, Thinktastic released ChatPoint yesterday.

It brings integrated, web-based chat to SharePoint 2010 (and actually would probably work fine in 2007 as well with some slight modifications).

Check it out and hit the contact form if you're interested.

Our goal is to eventually bring even tighter integration with GameTime so that you can get live notifications when things change in SharePoint (in addition to the basic chat functionality).

Filed under: Awesome No Comments

HTML5 on the Desktop — Finally!

Posted by Charles Chen

As tidbits start to come out about Windows 8, it appears that we'll see a significant shift in the Windows UI in the next version.  In itself, there's a whole lot of significance as it's a paradigm shift in terms of UI design.  But from a technology perspective, I find it more interesting.

Related to this issue is the question of how new-style applications will be written. Microsoft said that they will be HTML5 plus JavaScript, but again this precludes the possibility of migrating existing applications to the new interface. I'm told by insiders that HTML5 and JavaScript won't be the only option, and that existing applications (native, Silverlight, and WPF) will be migratable in some way, but specifics are still lacking at this time.

For a while now, I've been advocating against technologies like Flash and Silverlight for the web.

My stance has always been that aside from a few use cases (streaming media, 3D-graphics, complex animations, games), there are very few use cases that call for Flash or Silverlight.

WPF on the other hand, is a much more interesting and useful piece of technology, but even with WPF, I find that the development cycle is still slow and there are some idiosyncrasies with HTML that are hard to overcome for me (case in point is WPF Grid layout and HTML table layout).

Maybe because I come from a web background and because I've been writing web applications and building web interfaces since I was a teenager, I've always been able to build richer, more compelling, more interesting user interfaces in HTML -- faster -- than I've ever been able to do with WinForms or WPF.

So this is good news to me.  Time to get cracking on those HTML5 books!

Filed under: Awesome, Dev No Comments