<CharlieDigital/> Programming, Politics, and uhh…pineapples


Why Teachers Are _Really_ Leaving the Profession

Posted by Charles Chen

I caught a portion of an interview last night on NPR as I was in my truck, leaving Lowes.

I sat there for a moment, dumbfounded by what I was hearing and entirely outraged by the bullshit that Republican Wisconsin State Senator Glenn Grothman was spouting.

LYDEN: Last week, the Associated Press reported that nearly 5,000 Wisconsin teachers retired since the beginning of the year and that's more than half of the number from 2010. It's not a great way to start the year. Could the Republicans who passed this bill have done a better job of talking about it?

GROTHMAN: Well, I'm trying to talk about it right now. I will point out that at least one of the reasons why more teachers are retiring - we have heard anecdotal evidence that some of the worst teachers not waiting around for the inevitable and finally deciding it's time to retire. And I think some of those teachers who shouldn't have been there all along realized that without collective bargaining their time is up.

I was kind of riled up by this given that my wife is a former teacher (now an Learning Disabilities Teacher Consultant, but still works in the school system).  I've seen the dedication that she put into her work, I've seen the long hours grading papers and dealing with parents, I've seen results of her labor when parents stop her in stores years later and thank her for transforming her kids, I've seen parents specifically request to be placed into her class.

For all of this, she collects a relatively meager paycheck, but it's her passion and it's rewarding for her to be involved in the education of children -- in one capacity or another.  The low salary is made up for, somewhat, by the benefits that she receives.  Today, she came home with news about our insurance premiums: we're now required to pay about $280/mo. more for our health insurance plan because of changes enacted by Chris Christie and possibly the entire sum in a few years time.  To be clear, the sum of those premiums for a family of 3 is pretty much more than half of her monthly income -- enough to pay the mortgage on a small house.

I don't want to get into a long discussion on whether tax payers should or should not pay for the benefits of public employees, but at the end of the day, it is my belief that public employees are by-and-large hard working, middle class folk who provide valuable services to the taxpayers that are often hard to put a price tag on.  A good teacher can set a child on a path for success and help produce members of our society that contribute to our prosperity instead of being a drain on it.

But how can we possibly entice highly qualified candidates to these important jobs when as a society, we decide to strip these individuals of the benefits which largely make up for the otherwise less then illustrious labor of love?  Even those with a passion for teaching and helping transform kids will have to do a double take now and consider alternative career paths as we the movement against labor and the public sector continues to erode the middle class in America.

So I say to Glenn Grothman: the reason that teachers are retiring from the profession or quitting out-right is because society has created an environment where it's become untenable to put in the long hours and dedication required to be a good teacher for the already low salaries.  As the benefits that used to make up for the low wages continue to be eroded, the financial calculus will surely force more teachers -- especially the smartest and most qualified -- to reconsider their career choices for more profitable alternatives.

Post script: I think this is another case of Republicans driving the public sector to fail (by creating a dis-incentive for the most qualified, most skilled applicants) so that they can later point to the failure of the public sector!  Well, no shit, Holmes!  If you set the system up for failure, of course it'll fail.  Instead of the worst teachers leaving the profession, it's the best teachers that will leave the profession for better paying opportunities because these individuals are the ones that have the highest qualifications, the highest levels of professional ethics, and the highest ability to translate their skills outside of the classroom.

Filed under: Rants No Comments

A UX Pattern That Needs to DIAF

Posted by Charles Chen

Why, Home Depot, why?

There are more than a few shopping sites out there with these "fancy" picture viewers like Home Depot and Newegg.com.

This has to be one of the worst UX patterns out there and every site that uses it never fails to piss me off with aggravation just trying to get an idea of what a product looks like.

You see, these web designers, programmers, and architects seem to not realize that most browsers nowadays (and even a few years back at this point) have supported automatic resizing of images to fit the browser window with full-size "zoom" capabilities and that the built-in browser scrollbar is a perfectly acceptable method of navigating an image.

Better, but still a poor UX

Even worse in the case of the Home Depot site, the cursor shape over the image leads one to believe that the image can be clicked and dragged.  Instead, you can only drag and move the little red box on the right to pan the picture.

Newegg actually does the same thing, but it's a bit better because at the least, it doesn't popup in a new window in Firefox and to some degree, I can understand why they do it because of the gallery navigation and the 360-degree view (there's some "added value"), but I would appreciate the option of just being able to see the full image without it being cropped off within some silly window.

This pop-out, image viewer UX pattern is definitely one that needs to DIAF.

Filed under: Rants No Comments

Damn It, Microsoft (SharePoint)

Posted by Charles Chen

Came across an interesting quirk today with regards to AddFieldAsXml in the SharePoint API.

It turns out that SharePoint doesn't really give a damn what you want to call your field when it adds it to the list; it's going to damn well do whatever it damn well pleases!

You see, in the MSDN documentation, it implies that you can specify the display name and the internal name (just look at that sample code).

However this isn't the case and, as Bill Simser points out, this has been an issue going back to 2005.

Thankfully, as commenter Morten Bundgaard Pedersen points out, you can use the SPAddFieldOptions.AddFieldInternalNameHint option to force SharePoint to use the name value that you specify in the XML.

Seems kind of silly to me given that this is the most likely use case and should be enabled by default given that the intent -- in specifying the Name and StaticName fields would be to, you know, use them when creating the field.

Damn it, Microsoft.


Why I Can’t Be Bothered To Learn Silverlight

Posted by Charles Chen

Aside from hating Flash and Flash-like applications in general, as I've stated before, I view Silverlight as a dead/dying technology at the scope of web and business applications.

It may retain niche applications in gaming and the Xbox platform, but I think that's it.

Of course, Silverlight diehards will point their fingers hard at Windows Phone, but I say it really doesn't matter.  The trends seem to show that WP7 is dead in the water:

However, perhaps the most surprising finding of last quarter was that Microsoft's Windows Phone 7 platform was beaten out by Samsung's Bada operating system. According to Gartner, over 2 million Bada-based smartphones were sold last quarter, earning the platform 1.9 percent market share. Windows Phone 7 followed with 1.7 million units sold, helping it to earn 1.6 percent of the market.

The last 12 months have been difficult for Microsoft. According to Gartner, the company's market share just one year ago was 4.9 percent on over 3 million unit sales.

At an anemic 1.6% market share for 2Q11, one has to wonder -- from the perspective of a developer: why waste my time on a platform that has such a trivial market of users?  Silverlight is already dying in the wider web.  I can't remember the last time I saw a Silverlight enabled web page that wasn't a Microsoft property.   But with this news of WP7s declining market share, one has to wonder when the diehards, Silverlight defenders, and (the absolute worst!) the Silverlight evangelists will just give it a rest and stop trying to tell me how awesome it is and how every business wants Silverlight apps and all that nonsense.

With the news that Windows 8 will support HTML5+Javascript apps and a tablet- and touch-friendly interface, one has to wonder how much longer Silverlight will have applicability in the mobile space.  Could Microsoft scrape the current, Silverlight-driven Windows Phone platform for one based on HTML5 as a derivative of their Windows 8 platform?

One thing is for certain, the market has spoken: Silverlight sucks!  Now go away, please (and take Flash with you!).

Filed under: Rants 6 Comments

When is InfoPath Ever the Answer?

Posted by Charles Chen

Since I've started working with SharePoint back in beta 2 of 2007, I've wondered what the infatuation with InfoPath is.

Everyone from enterprise architects and business users seems to be very intrigued by it.

As a more technical guy who's had to dig into it, I find it a puzzling beast and really don't understand why folks even give it a second whiff.

To understand why, let's take a step back and examine the landscape of forms technologies which are available in the typical enterprise environment and how they fit into SharePoint.

Web Forms

  • Pros
    • Extremely flexible and powerful; a developer can create any form imaginable.
    • People are familiar with web forms because they use them: Every.  Single. Day.  When they log on to their gmail, when they buy something from Amazon, when they sign up for Facebook, when they log into The New York Times website -- people use web forms every day.
    • Deploys fairly easily into a SharePoint environment using .wsps and can be permissioned like other SharePoint artifacts.
    • Easily understood by developers and easy to hire developers or contractors to manage; web forms developers are a dime a dozen.
    • Easy to integrate additional data into the forms since it's easy to call out to web services, look up files in a file system, perform searches across SharePoint, etc. to incorporate additional data and intelligence into your forms.
  • Cons
    • Requires a developer to create and update the form.
    • Requires custom programming.
    • Not available on the desktop or offline.
    • Doesn't work well for forms that need to also support print versions since it's nearly impossible to create a printed layout that will 100% match the on screen layout.

Word Document Forms

  • Pros
    • Fairly flexible with a programming model that can be easily accessed by power users via record macro or custom written macros.  Can build fairly complex interactions.
    • Deploys easily into a SharePoint environment by simply uploading a document template to a list and associating it as the default template.
    • Quick Parts map fields directly onto content type fields on the list - save the document back to SharePoint, and the fields are updated in SharePoint.  Update fields in SharePoint and they are pushed down to the Word document.
    • Business users can create these easily.  In fact, if you asked someone in HR -- for example -- to create a form, more likely than not, they will create a Word or Excel based form.
    • Form can be completed offline and saved back to SharePoint.
    • Printed form is pretty much identical to electronic form.
    • Businesses already have large inventories of Word forms.
  • Cons
    • Macro programming isn't accessible to most business users.
    • Macros may present a security risk.
    • Not as easy to perform complex tasks (for example, multiple web service calls) as it is in web forms.
    • No direct integration with the server environment and the SharePoint content store - may still need event receivers and what not to accomplish certain tasks.
    • Requires Word runtime on the desktop.

PDF Forms

  • Pros
    • Form is "locked down" once deployed.
    • Form is easy to build using Adobe Acrobat.
    • Printed form is identical to electronic form; create one form and use it everywhere.  This is especially useful in some cases in some industries like life sciences.
    • Everyone has Acrobat on their desktops.
    • Business users already have the skills to work with Adobe PDF forms and PDF forms can be easily created from Word forms.
    • Businesses already have large inventories of PDF forms.
  • Cons
    • Requires paid versions of Acrobat to create forms, but your business users probably already have these.
    • Requires some custom programming to expose the form fields to SharePoint (but easy to program and only needs to be done once); not supported out of the box.
    • Dynamic forms are difficult to create, requiring developers to write embedded Javascript or logic to build the form dynamically on the server before serving it.

InfoPath Forms

  • Pros
    • Microsoft really wants you to use it and thus they've rigged it into various points in SharePoint, Office, and SharePoint workflows (but you can do these with web forms, too).
    • Some forms can be used in the web or on the desktop.
    • Easy to build basic forms (but not as easy as it is to use Quick Parts in Word).
  • Cons
    • Requires license to InfoPath to create forms and your business users probably don't have these.
    • If you ask a business user to create a form, the user will either: 1) create it in Word, 2) create it in Excel, 3) create it in PDF, 4) ask IT to create it.  No business user will instinctively think "Ah, I'll create an InfoPath form".  It's not a technology easily leveraged by anyone but someone technical.  The idea that power users can create and own their forms is a myth; no business user wants to spend their week working on a form when they have hamsters in IT to do that for them.
    • With that said, it's going to be a LOT easier to find developers who are strong in web forms (and they'll probably be cheaper) than it is to find developers who are strong in InfoPath (and they'll probably require higher rates).
    • While some forms can be used via Forms Server or on the desktop, the truth of the matter is that Forms Server enabled forms are limited to a subset of the desktop forms' functionality.
    • InfoPath forms are difficult to work with programmatically compared to writing Word macros or ASP.NET web forms.
    • Anything beyond basic interactions will require programming anyways, so why not pick the technology that's easier to program for?
    • Complex form fields (for example, repeating rows) don't map to SharePoint fields and cannot be surfaced in SharePoint as metadata, making it somewhat less useful unless you're willing to spend the effort to manually manage that serialization of data.  But then, why would you choose InfoPath and not Word or web or PDF forms?
    • Electronic forms are not identical to printed versions (as is required for some FDA forms which can be submitted electronically or as paper forms or as scanned versions of paper forms) so in some workflows, you'll need to create separate forms for printed versions and consolidate that data manually.  If you need to create those printed forms in Word or PDF anyways, why bother creating InfoPath forms?
    • Difficult to share forms with users outside of your corporate firewall because: 1) they probably won't have InfoPath on their desktop (but they will have or can easily get Acrobat or Word) or 2) you have to get them access to your network to serve them the form via Forms Server.  With Word based forms or PDF forms, you could theoretically just email the forms, collect them, and upload them -- it gives you the flexibility to create these types of workflows.

Ultimately, I think InfoPath is a poor choice for forms because there are better options either from the perspective of developers or the perspective of a business user.  It's a myth that InfoPath will allow business users to self service and easily create and customize their own forms and thus save time and money.  If anything, it's even easier for business users to create their forms in Word, a tool they are already intimately familiar with.  Anything beyond the most basic of forms will require programming and IT involvement anyways and if that's the case, why bother with InfoPath when it's harder to resource for?

I don't think I'll ever understand the infatuation with InfoPath as a solution.  I do wonder if most folks are simply unaware of the Quick Parts functionality in Word 2007 and 2010 and how nicely it integrates with SharePoint.

Filed under: Rants No Comments

Responsibility in Consultancy

Posted by Charles Chen

As a consultant, I feel strongly about giving sound technical advice to my clients, even if such advice means saying "no" to a client or possibly turning back a larger project for a more pragmatic one. It's about doing the right thing and offering sound technical advice to the best of my knowledge -- not just money, projects, and utilization.

The one personal example that really sticks out for me is the case where Microsoft sold a deal to a hedge fund to build a bulk import system using BizTalk that would have cost them triple the price (once licensing and hardware was factored in) of doing it using SQL Server DTS, which was easier to program, maintain, and more robust in every way (not to mention this company already had SQL Server skillsets in-house).  Luckily, we were able to convince the client that DTS was purposefully designed for carrying out bulk import and transform of data before they committed the cash to BizTalk.

Recently, a friend of mine showed me a project that the Big Consulting Company he works for was delivering to their client, a public library. It looked really good for a public library website...until he dropped the bomb that it was built using Silverlight (and to top it off, he was really proud, too -- as if I was supposed to find it impressive).  I don't think I've ever done a bigger facepalm in my life.

As I've stated in the past, I have a strong disdain for the misuse of Silverlight.  There are certainly scenarios where it should be used for building web sites:

  1. Streaming media
  2. Scalable 2D vector graphics and animation
  3. 3D graphics and animation
  4. Interactive games

And that's it!  Beyond that, if a company wants to use it in their intranet site, it doesn't concern me as much because the environment is more homogeneous and controlled in terms of having the platform to run the Silverlight applications; it's their headache going forward.  Besides, if it's a private, multi-national company, then by all means; if they wish to waste their capital and resources, that's their choice.

However, it is a damn crime to recommend Silverlight to any client building basic web applications that are Internet facing, especially a public library financed by taxpayers.  I mean, people should be fired and embarrassed for offering such terrible advice.  To begin with, few non-Windows devices natively support Silverlight (and even folks on older Windows OSes can't natively run Silverlight apps).  iPad?  iPhones?  Android phones?  Linux based netbooks?  As sales of traditional laptops and desktops decline, it's important to factor in the presence of these newer platforms when designing a publicly facing Internet site.  I would think that this would be even more important for a public library.

Now, if the site were media focused -- like a YouTube -- perhaps it could be forgiven; after all, HTML5 is still a moving target and supported only by newer browser versions.  But this is a public library website that was listing books...It's as bad as websites that still use Java (yes, Java without the "Script") for image galleries or raindrop effects.  It's as bad as websites using Flash for menus and menu rollover animations.

I would be embarrassed to be a part of the company or the team that sold and implemented this deal.  A fucking crime to the taxpayers of the township with me as the perpetrator; no better than stealing money from your neighbors.  I couldn't live with myself for being so evil.

Now, he told me that the client insisted on Silverlight and that it was they who wanted it done in Silverlight.  To me, that makes no difference.  As a consultant, it's my duty to provide sound technical guidance to the best of my knowledge and ability.  If there is a more compatible, cheaper, easier to maintain solution built on a platform with greater longevity that solves the same problem, I will recommend taking that route, even if it takes me out of the running.  It's our job as consultants to consult and to offer sound technical advice.

For you see, the client may not know or care for the difference between Silverlight and HTML5 or jQuery based UIs.  The client may be under the impression that a given UI or bit of functionality is only possible because of Silverlight if that's what they've been sold and demo'd.  The client may not understand the alternative solutions as certainly, for a non-expert, the difference between two types of wood -- for example -- aren't perceivable.  The client may be enamored with one buzzword or technology, but it is our duty and responsibility as consultants (and decent human beings) to tell the truth because I'd like to believe that when I ask a contractor to come to my house for a quote or get a diagnosis from an auto mechanic, he'd do the same for me and give me the low-down to the best of his or her ability and knowledge.

In the end, I was so peeved by what my friend had shown me, I took 30 minutes and rebuilt the same exact functionality that they had implemented in Silverlight using nothing but jQuery and CSS with only 20 lines of JavaScript and 5 lines of CSS after being challenged to do so.

I'm still peeved by this as it's a critical misunderstanding of the Internet ecosystem and managing device compatibility as well as a critical misunderstanding of technology and their suitability for a purpose.  Not to mention that it's a terrible choice for audience accessibility, long term costs, and maintenance.  I really don't want to be upset by the fact that my friend or his team could have purposefully offered bad advice for greater financial returns as that would be a true embarrassment and I only hope that all sides in this come to their senses and ditch Silverlight.

In the end, for me, consultancy is about people and treating customers with respect by offering the best technical advice to one's knowledge.  Even if it costs me my job, I've always believed that I am accountable to my clients and I'm responsible for giving sound technical advice.

Filed under: DevLife, Rants 2 Comments

Commentary on “Frankenfish”

Posted by Charles Chen

The news media has recently been abuzz about about this so-called "Frankenfish".

It's been puzzling to me what the hullabaloo has been all about.  The fact of the matter is that humans have been altering the genetics of just about everything we eat for centuries (millennia?).

Those navel oranges you eat? They're all genetic clones of a single mutation that occurred in the 1800's and every navel orange since has been grown via cutting and grafting techniques. Most cultivars of avocados are also grown via cutting and grafting of a single plant with a desirable genetic mutation.  That bread you eat? It's probably made from wheat that's been bred and cross-bred for resistance to certain strains of fungi and resistance to insects.  The corn that you eat (and all of the byproducts made from that corn)? It's been bred, cross-bred, and selected for desirable traits for centuries.

Humans have been manipulating the genetics of the food that we eat and disrupting or enhancing the natural reproductive cycles of plants and animals alike to breed for desirable traits like pest resistance, drought resistance, fatter meat, leaner meat, tastier meat, greater milk production, faster growth, sweeter fruit, and so on. And when genetics aren't enough to get the desired results, humans aren't shy to rely on other aspects of science like artificial steroids, hormones, antibiotics, pesticides, fungicides, herbicides, and so on. All of which make it into our waterways and our digestive systems.  I'm not saying that these are "good", but that the food supply that you already eat from is hardly free from human intervention.

The rampant and unjustified fear around genetically modified food is symptomatic of a culturally ingrained distrust of science (maybe it stems from religiosity...) and a general ignorance about agriculture and the long history of selection and crossbreeding for genetic traits. Certainly, we can't simply take AquaBounty's word, but we can approach this rationally and study the science and study the data to ensure that indeed, the inserted genes do not have an undesirable side effect in humans and that the environmental impact will be safe.

In the big picture, this type of science is needed if we wish to responsibly address the growing population of the Earth. Our natural resources aren't getting any more bountiful, yet the human population continues to grow, devour, want, and so on. If genetic modification can help yield greater harvests from the same land, if genetic modification and result in the decreased use of pesticides or fungicides or herbicides or fertilizer, if genetic modification can make farm raised fish profitable and thus help the recovery of wild salmon stocks, if genetic modification can help feed the growing population of the Earth and decrease famine and hunger, then I ask why should we not embrace this science and find solutions that work?

It recalls the criticism that Norman Borlaug's work received:

Borlaug's name is nearly synonymous with the Green Revolution, against which many criticisms  have been mounted over the decades by environmentalists, nutritionists, progressives, and economists. Throughout his years of research, Borlaug's programs often faced opposition by people who consider genetic crossbreeding to be unnatural or to have negative effects.

And yet, Borlaug's work has arguably saved billions of lives:

Borlaug received his Ph.D. in plant pathology and genetics from the University of Minnesota in 1942. He took up an agricultural research position in Mexico, where he developed semi-dwarf, high-yield, disease-resistant wheat varieties.

During the mid-20th century, Borlaug led the introduction of these high-yielding varieties combined with modern agricultural production techniques to Mexico, Pakistan, and India. As a result, Mexico became a net exporter of wheat by 1963. Between 1965 and 1970, wheat yields nearly doubled in Pakistan and India, greatly improving the food security in those nations.  These collective increases in yield have been labeled the Green Revolution, and Borlaug is often credited with saving over a billion people worldwide from starvation.

Of course, there are many that decry humanitarianism as being far from the objectives of AquaBounty; they say that AquaBounty is only in it for greed, for money, for profit.  But then I have to ask: what commercial fishing or farming operation isn't in it for money and profit?  Even that mom & pop organic farm down the street is in it for profit.  It's the very basis of the capitalistic system: do more, cheaper, faster, more efficiently.  There's no such thing as food production that doesn't follow this basis (with few exceptions like the production of fine liquors or wines, for example).

Certainly, there are pitfalls and certainly, there are dangers. However, the reality is that many wild fish populations are being fished to the edge of extinction or will be fished to the brink of extinction if we don't take responsible action today. That includes developing better systems of quotas and monitoring of natural populations, decreasing pollution in our waterways, and developing alternatives that can alleviate the strain that commercial fisheries place on these populations.  In the broader picture, if we also consider land based crop farming, improving efficiency through genetic engineering may be necessary to curb deforestation and the continued destruction of natural habitat while still meeting the nutritional needs of a growing population.  Borlaug developed a hypothesis with regards to the importance of increasing yields through science:

The large role he played in both increasing crop yields and promoting this view has led to this methodology being called by agricultural economists the "Borlaug hypothesis", namely that increasing the productivity of agriculture on the best farmland can help control deforestation by reducing the demand for new farmland. According to this view, assuming that global food demand is on the rise, restricting crop usage to traditional low-yield methods would also require at least one of the following: the world population to decrease, either voluntarily or as a result of mass starvations; or the conversion of forest land into crop land. It is thus argued that high-yield techniques are ultimately saving ecosystems from destruction.

I deem these fish safe until the science tells me otherwise. For all intents and purposes, they've only inserted genes from two other fish species (one of them being another type of salmon!) for their desirable traits; hardly worth the shock response and uproar over these GM salmon. The "Frankenfish" label is completely based on ignorance and stoking the fears of the ignorant.


Meeting Hell

Posted by Charles Chen

From one of my favorite software engineering books, Eric Brechner's I.M. Wright's Hard Code:

None of us is as dumb as all of us

An especially evil form of interruption is the meeting.   A meeting forces you to stop productive work and throw yourself into a frustrating, time-consuming, black hole of wasted life from which you can never recover.  However, there are several actions you can take to minimize the life-draining effect of meetings:

  • Stop going.  There are only a few meetings you must attend: your on-on-ones, your project status meetings, and your staff meetings.  Almost all other meetings are discretionary.  If a meeting feels optional, try skipping it.  If nothing bad happens, don't go again.
  • Make someone else attend.  Try delegating the meeting to someone else.
  • Run effective meetings.  As for meetings that are left, make them as effective as possible.
  • Put all your meetings back to back.  I know this sounds strange, but the idea is to reduce context switches.  First, try to schedule your project and staff meetings early in the week, and then schedule all your one-on-one meetings around them.  Sure, the early part of your week will be hellish, but the middle and the end of your week will have uninterrupted blocks of time.

Sage advice.

Filed under: Rants No Comments

The Math of Mediocrity

Posted by Charles Chen

Professionally, almost nothing aggravates me more than the Math of Mediocrity.  The only thing worse than observing failure based on the Math of Mediocrity is having to actively participate in it.

Steve Jobs’ Parable of the Concept Car is a perfect illustration of how companies fail to execute because they fall for the Math of Mediocrity:

"Here's what you find at a lot of companies," he says, kicking back in a conference room at Apple's gleaming white Silicon Valley headquarters, which looks something like a cross between an Ivy League university and an iPod. "You know how you see a show car, and it's really cool, and then four years later you see the production car, and it sucks? And you go, What happened? They had it! They had it in the palm of their hands! They grabbed defeat from the jaws of victory!

"What happened was, the designers came up with this really great idea. Then they take it to the engineers, and the engineers go, 'Nah, we can't do that. That's impossible.' And so it gets a lot worse. Then they take it to the manufacturing people, and they go, 'We can't build that!' And it gets a lot worse."

When Jobs took up his present position at Apple in 1997, that's the situation he found. He and Jonathan Ive, head of design, came up with the original iMac, a candy-colored computer merged with a cathode-ray tube that, at the time, looked like nothing anybody had seen outside of a Jetsons cartoon. "Sure enough," Jobs recalls, "when we took it to the engineers, they said, 'Oh.' And they came up with 38 reasons. And I said, 'No, no, we're doing this.' And they said, 'Well, why?' And I said, 'Because I'm the CEO, and I think it can be done.' And so they kind of begrudgingly did it. But then it was a big hit."

This doesn’t just happen in automobile manufacturing or engineering, but it also occurs plenty often in software development.  One particular example is what I call the “Communism of Failure”.  In this case, the best solutions, the best ideas, the ones that will benefit the end users the most, the ones that will get the users excited, the ideas and solutions that will help people be more productive or more efficient are…shelved.  Why?  because it can’t be supported by the commoners in tech support.

Certainly, this is a valid concern -  it would be absolutely foolish to believe otherwise; a solution is useless if only the brightest and most exceptional of minds can understand it, deconstruct it, rebuild it, fix it and so on.  But proper software engineering and project management offers ways to mitigate this through process and practice.  Pair programming, strict guidelines and expectations for documentation, well documented common coding styles and techniques, leveraging automation and code generation, encouraging reuse of code assets by thinking in terms of frameworks.  The point is, building an exceptional solution is not exclusive of building a sustainable solution.

The Legalist philosopher Han Fei-Tzu wrote:

If it were necessary to rely on a shaft that had grown perfectly straight, within a hundred generations there would be no arrow. If it were necessary to rely on wood that had grown perfectly round, within a thousand generations there would be no cart wheel. If a naturally straight shaft or naturally round wood cannot be found within a hundred generations, how is it that in all generations carriages are used and birds shot? Because tools are used to straighten and bend. But even if one did not rely on tools and still got a naturally straight shaft or a piece of naturally round wood, a skillful craftsman would not value this. Why? Because it is not just one person that needs to ride and not just one arrow that needs to be shot.

Indeed, a solution catered towards the brightest of minds is equally as bad as a solution catered towards the most common of capabilities.  But through the usage of the right tools and the application of bending and straightening, we can bridge the two.  It's not a compromise, but rather an effort to build a framework that allows excellence to propagate.

As in Jobs’ parable, where solution and enterprise architects fail at the Math of Mediocrity is that they cede to the concerns of the plebeians; they sacrifice excellence for “good enough” so that tech support can support the solution.  On the one hand, Solution A can solve the problem for the end user in 1 click.  On the other hand, Solution B requires the end user to perform more than 20 clicks to complete the same operation but is easier for tech support.  Which is better?  Jobs’ would surely side with Solution A, even if it’s the more technically complex solution because it delivers a better user experience and improves efficiency and productivity.  Amazon loved Solution A so much that they gave it a name and patented it.

The problem arises because of ambiguity in calculating the true cost of Solution A compared to Solution B.  Solution A may require $300/HR consultants and twice as much time to implement.  Solution B may require $150/HR consultants and cost only half as much as Solution A.  These costs are concrete and easy to quantify and grasp.  What seems to escape the calculus is the cost of lost productivity and efficiency by having the end users, who have to use the applications day in and day out, suffer through an inferior solution because of the marginal costs of contracting better developers, working smarter, building from a framework, and hiring more competent tech support.

The same math carries over to the support staff, who are a marginal percentage of the overall workforce for any large organization or for a particular project or initiative.  The question is whether it’s better to hire more competent support staff who can maintain a more complex but better solution or to hire less competent support staff at lower costs.  And again, the question comes back to the issue of productivity and efficiency and comparing the gains made across an organization of tens of thousands of people versus the additional costs associated with a few dozen people.  No question: I improve the user experience which not only improves productivity and efficiency if done right, but also aids in adoption and uptake;  cost is but only one metric of success and perhaps not even the most important one.  In the end, it really doesn't matter how much you saved by approaching the problem using Solution B; if the end result is a clunky, hard to use, inefficient, productivity drain, then the project has failed, regardless of how much money was saved by catering to the mediocre.

A solution architected and designed around a compromise for the average can work, but the problem must be approached differently.  Leverage better developers from the get-go, make documentation a priority, standardize code, leverage patterns, ensure that the right tools and platforms are in place, build frameworks to support the most common scenarios, use pair programming and code reviews to ensure cross pollination of skills and knowledge, make learning and education a primary job conern, find solutions through engineering and process, not through capitulation to the lowest common denominator.

Update: An article in the New York Times by Damon Darlin this weekend caught my attention and adds another layer to this:

Remember those lines? Back when commissars commanded the Soviet Union’s economy like Knute commanding the tides, people would wait for hours in long queues for free bread. Although the bread was free, people paid for it with their time.

To economists, the long lines were a real-life example of the market requirement that payment be made one way or another — in money or in time. (In this country, the long lines would be for an Apple gadget, which is neither cheap nor scarce. But explaining that mystery is for another time.)

Paying with time rather than money seems just as common on the Web...Technology could very well make the Soviet bread line disappear. Do you remember how long it took to do a Google search a dozen years ago, when the service started? Probably not, but Google engineers calculate that their refinements have saved users a billion seconds a day. Using Google to quickly make the calculation, that comes out to about 1,800 lifetimes.

Indeed, the question is whether a business wants to make payment in money or in time (well, for businesses, time is money).  For that reason, enterprise architects should think long and hard about the priorities of the platform or solution they are architecting.  Is it just to do the minimum and keep maintenance effort and costs low?  Or is it to actually streamline and improve the business processes and improve productivity and efficiency?  How can you achieve the latter while sacrificing minimally of the former?

Filed under: DevLife, Rants 2 Comments

Why SPMetal Falls Short

Posted by Charles Chen

First, SPMetal is good.  It's very good.  Much better than life without it.  It encourages more object-oriented programming (instead of XML string oriented programming - blech!)

That said, SPMetal falls short of awesome by just a hair.

Ideally, one would be able to generate models from local CAML files instead of having to deploy the content types first since I assume it's best practice for organizations to deploy the content types from CAML files for consistency.  It seems backwards from a development perspective to have to deploy your content types and then be able to generate models to write code.

Filed under: Rants, SharePoint No Comments