<CharlieDigital/> Programming, Politics, and uhh…pineapples


An Alternate Meaning for FOCKED

Posted by Charles Chen

Eric Brechner came up with one of my favorite acronyms of all time in software development: FOCKED.


I want to add an alternate: Failure to Orchestrate Collective Knowledge Effectively for Delivery.

Successful delivery of software requires that different members of the team come together and understand the goals that have to be achieved and the priorities of those goals.

It's as simple as communicating to the team on a regular basis (no more than once a week, but maybe at least once a month):

  • where we are,
  • where do we want to go,
  • when do we have to get there,
  • how are we getting there,
  • who's driving

It can make the whole process of delivery of software much less stressful and maybe more successful simply by aligning all of the stakeholders periodically.

Hey, maybe you learned this in some fancy MBA class or something, but I'm starting to appreciate -- more and more -- that the real secret to successful delivery of software is driving the successful collaboration and communication of people and alignment of all pieces to a strategy, vision, or goal.  Having a bunch of smart, capable people doesn't help you much if no one knows what's going on.

Filed under: DevLife, Rants No Comments

Adventures in Poorly Written APIs

Posted by Charles Chen

I'm working with a library that I have been fighting with for the better part of three days now to try to get it to work.

The previous version of this library was actually very well written from an API perspective and well documented.  The new version?  Not so much.

But from this, I take away two lessons in API design.

Lesson 1: Don't Use Dictionaries in Public APIs -- ESPECIALLY for Required Data

The first mistake the API designers made in this case was exposing a "dictionary" in the public API.

This itself may not be all bad if the API is highly flexible and can accept many forms of requests, but if a public dictionary interface is required, then at least have the good judgement to provide a wrapper to populate those dictionaries.

But even the lack of usability is not as bad was having required data inputs passed to your service as dictionary entries.  In other words, the service call won't succeed without some sort of magical recipe of just the right metadata keys and values.  If it's a required piece of metadata, why not surface it as an actual parameter to the function?

Exposing dictionaries in public APIs is a terrible design idea which is as bad as APIs that require you to pass in XML strings (understood if it's a must, but have the courtesy to provide downstream callers an API to serialize objects into that XML and don't put required data into open ended inputs).  They should be used infrequently and preferably wrapped by an abstraction layer.

Lesson 2: Don't Require Your Caller to Keep Passing You Data You Handed Back

In this particular API, there is an initialization handshake with the server whereby large bits of system data are passed back to the caller.

For whatever reason, instead of simply handing a token back to the server, all (or part?? -- again, it's a magical incantation of sorts) of this metadata must be exchanged with the server to complete a request!

Why not provide a session key?  A caller token?  Something that the caller can use to represent this collection of system data instead of the full set of system data?

More to come, I'm sure of it, but this has been an interesting adventure in poorly written APIs.

Filed under: Dev, Rants No Comments

The Dichotomy of Change Control and Quality Software

Posted by Charles Chen

When processes attack!

When processes attack!
(Sybren A. Stüvel via photopin cc)

It may seem counter-intuitive to the old guard that change control can actually negatively affect the quality of software, but when implemented in such a way that it dramatically increases the cost and effort of making software improvements, it leads a team down a path of mediocrity.

You see, good developers naturally want to fix things that they find wrong with the software.  Tweak it to be a little faster.  Make a button bigger so it's easier to see.  Change the alignment of some form labels.  There are endless ways that good engineers will reflect upon their work or the work of their peers and find better ways to deliver a superior user experience.  Good teams will also find ways to quickly fold user feedback into the UX design and improve the usability of the system.

Indeed, there are times when changes are drastic and require that proportionally sufficient effort is put into testing, but there are times when changes are small and well contained where running the same process for big fixes simply yields frustration and drives a team to ultimately "settle" for fear of waking the slumbering giant of change control.

Consider a doctor performing open heart surgery.  Of course, the protocol for sanitation and safety checks are going to be far more rigorous than for administering a flu shot.  But this makes sense because the risk is far, far greater with one than the other.  If a doctor's protocols for preparing for surgery were one-size-fits all and applied for administering a flu shot, I suspect the quality of care in the US would decline drastically while causing the cost to skyrocket!  Or consider the permits and documentation required if you want to add a walkout entrance to your basement.  Of course you will need engineering specs, signed permits, and so on because the risk is high.  No one would think of requiring certification from a structural engineer and permits from your municipality to hang a picture frame on your wall because it would be absurd!

Likewise, common sense, pragmatism, and a sensible balance is a requirement for any software quality process; a one-size-fits-all approach creates a wall of effort for even minor tweaks that simply means that the small fixes that can make a big difference in the usability and functionality of the system aren't made for fear of generating a ton of e-paperwork.  The Broken Windows Theory, popularized by Steven D. Levitt and Stephen J. Dubner in Freakonomics, explains this phenomenon:

In an anonymous, urban environment, with few or no other people around, social norms and monitoring are not clearly known. Individuals thus look for signals within the environment as to the social norms in the setting and the risk of getting caught violating those norms; one of those signals is the area's general appearance.

Under the broken windows theory, an ordered and clean environment – one which is maintained – sends the signal that the area is monitored and that criminal behavior will not be tolerated. Conversely, a disordered environment – one which is not maintained (broken windows, graffiti, excessive litter) – sends the signal that the area is not monitored and that one can engage in criminal behavior with little risk of detection.

In the education realm, the broken windows theory is used to promote order in classrooms and school cultures. The belief is that students are signaled by disorder or rule-breaking and that they, in turn, imitate the disorder.

When change is made expensive, niggling things here and there in the code and UX that don't get fixed simply accumulate over time and promotes discord and disorder.  When the processes discourages innovation and excellence, a team ends up simply mired in mediocrity with no real quality to show for it.  It's great that there's now a trail of paperwork long enough to circle the Earth for the changes that were made, but the real shame are the fixes, improvements, and ideas that weren't implemented because of the cost in paperwork.

Again, this is not to say that no quality processes or change control should exist, but that their application must be proportional to the risk and pragmatic about the priorities of the project.

One way to solve this problem is via rigorous automation for it enables teams to condense the processes into robotic automatons that take minutes, not hours to execute.  It enables teams to conceptualize, develop, and deliver more rapidly through continuous deployment.

Wired had a good article on how LinkedIn is able to build and release new features quickly:

LinkedIn is a Wall Street darling, its stock up more than threefold in two years on soaring revenue, spiking profits, and seven straight quarters beating bankers’ estimates. But LinkedIn’s success isn’t just about numbers: an impressive acceleration of LinkedIn’s product cycle and a corresponding revolution in how LinkedIn writes software is a huge component in the company’s winning streak.

Much of LinkedIn’s success can be traced to changes made by Kevin Scott, the senior vice president of engineering and longtime Google veteran lured to LinkedIn in Feb. 2011, just before the buttoned-down social network went public. It was Scott and his team of programmers who completely overhauled how LinkedIn develops and ships new updates to its website and apps, taking a system that required a full month to release new features and turning it into one that pushes out updates multiple times per day.

By creating the tools and process and training the team to operate under continuous deployment, LinkedIn was able to quickly bring concepts and ideas to life; it is because the cost of making changes and improvements to the software have become cheap that they can be made readily and without friction.

True software quality can never be achieved solely through heavy-handed processes (unless they are automated!); such processes are simply paper tigers that lead to the appearance of conformance but instead, create an impedance to creating better software.


Microsoft, SharePoint, and Enterprise Social

Posted by Charles Chen

Every once in a while, I'll talk to a colleague and the topic of SharePoint and "social" will come up.  I mean literally, every iteration of SharePoint adds some other "social" feature or another.  It always leads me shaking my head.

You see, Microsoft really talks a great game when it comes to social and SharePoint.  It's the next big thing!  It's all new!  Look at all this cool stuff that you can do!

But all you have to ask yourself is whether Microsoft has proven themselves in any aspect of social.  The answer is simple: no.  And social for SharePoint is no different.

There is a very fundamental reason why it's a non-starter: social for the enterprise is stupid.

"But doesn't NewsGator make millions of dollars?"

Sure, but social for the enterprise is still stupid.

"But didn't you hear?  They purchase Yammer!"

Yes, but social for the enterprise is still fundamentally broken.  Never forget: Google once offered Groupon $6 billion -- doesn't mean that it was ever a good idea.

At the core of why social and the enterprise is fail, it's actually pretty simple as it comes down to one main issue: Portability.

Of course, for most people, their social networks often include a nexus around co-workers and professional acquaintances.  The perfect case for social for the enterprise, right? Wrong!  Because LinkedIn already has that covered and, more importantly, a professional social network on LinkedIn is portable.  If I leave the company, I can still stay in touch with all of my contacts and connections at my previous companies via LinkedIn whereas my investment in a closed network is entirely thrown out the window the moment I leave a company.

A closed-system, non-portable social network is a useless one; it's a non-starter.

Aside from the major flaw of lack of portability, there are a few other shortcomings that come to mind.

The first is that, in a large company, I simply don't care about 99.5% of the people in the company; I'm only connected to these people because we happen to be employed by the same company, but that could change at any moment.  In a small company, I will already have everyone added in LinkedIn or Facebook.  What good is a closed-system social network when the people in that network would largely be people I already work with on a daily basis?

The second is that, in my opinion, what is of more value to teams are not social capabilities, but collaboration capabilities.  Social is like the antithesis of task-oriented collaboration capabilities.  Teams would get more value out of an investment collaboration capabilities than an investment in social capabilities.  What I need from my enterprise platform is not social, but tools to help me and my team get things done.  You want to know what other people are working on, but usually in the common context of a project or such so tools that are focused on communication and collaboration in the context of project workgroups seems much more useful and practical an investment to me.

The third is that many companies include a forum or discussion type system in their social initiatives.  I saw this at CSC where Jive was deployed.  But this kind of circles back to the closed-system problem: if I have a question or if I'm working on a hard technical problem, am I more likely to find an expert within my company?  Or am I more likely to find an expert within the global community?  This approach makes sense maybe within Microsoft or Google, where the concentration of highly specialized subject area experts is high.  But in a consulting company, for example, it is less likely to be of any value as the global community will include a greater breadth of subject matter experts with a more extensive depth of knowledge.

Microsoft and social -- especially in the enterprise -- is pretty much a pointless endeavor, in my opinion.

Filed under: Rants 2 Comments

Avoid West Elm

Posted by Charles Chen

It's hard to imagine how some e-tailers are still functioning like it's 1999 in the age of Overstock and Amazon.

I ordered an item from West Elm online on October 18th assuming that I'd have it in my possession within a week.

Nope!  A week later, and nary an email on the status of the order, which is still listed as "shipping soon" on their website.

Turns out on 11/09, I'll get a call to set up a delivery date....

I don't even get the privilege of setting up a delivery date until almost a full month later?  Are master craftsmen hand-crafting my headboard, right at this very moment?  Given the reviews of their quality (or lack thereof), I think not.

I'm not sure if their stock management system sucks or what but this seems pretty crazy (OK, I'm spoiled by Prime as well).  I think it's a bit silly that I had to contact a live person to get a status on my delivery.

Turns out that this is SOP for West Elm and would have avoided them from the start had I known.

Filed under: Rants No Comments

Oh, Visio, How You Rustle My Jimmies!

Posted by Charles Chen

I have been working on putting together an information architecture/file plan document which shows the layout of artifacts in our SharePoint environment and came across this nice little gem after spending a bit of time putting together my stencils:

As you can see, this is the multi-connector and it seems to work great.  But then I realized that it seems that the multi-connector only allows for 6 branches...

Amusing and frustrating because it makes no logical sense to me.

Filed under: Rants No Comments

Windows Azure “Introductory Special” — WTF?

Posted by Charles Chen

WTF Microsoft? Seriously?

This is what you call an "introductory special"?


This is fine and all, except in the months of January and February -- in fact, since I've signed up -- I've never used the caching service.  Heck, I've barely used Azure!  How can I possibly be billed for $46?  Yeah, I'm pretty much cancelling my sub after this; can't waste my time and energy every month double checking to see if I'm being ripped-off by Microsoft.  And on top of all that, I have to wait 4 hours for a support callback...

Now here is my theory: I am being billed $4/day just for having 128MB of cache allocated, whether I use it or not.  But this is beyond fucking stupid. It hurts adoption because it drives developers -- like me -- away from even signing up in the first place if I have to police by billing, even when I'm not using the service.  And seriously $46/mo. for having 128MB of web accessible memory available?  I get more memory allocated from my $7/mo. hosted shared server.  WTF?

I can accept the cost if there was some way to opt out or if there was some notice that I would be billed for being allocated the cache space or if there was some à la carte way to turn it off -- my mistake for not noticing in those cases.   But I was not given an option to opt out of cache, I was not notified that I would be billed regardless of actual usage, and as far as I can tell, there is no way to turn off cache.

This is some serious bullshit.

Filed under: Rants No Comments

Airline Boarding and How to Make It Suck Less

Posted by Charles Chen

Caught an unexpected segment in the news on airline boarding (ending is great):

I've mostly had (terrible) experiences with boarding Continental flights and I've often thought why the process is so screwed up.  I've come up with the following conclusions myself:

  1. Problem: Last time I flew out of John Wayne International, literally 3/4 of the flight was "Elite" status.  So when they did the pre-boarding, it was a huge throng of people boarding the plane at once, which negated the whole point of boarding the plane from the back first. There's something broken about the system here because it creates a weird paradox.  Being able to board the plane faster is a "benefit" of Elite status, but by using this system, it extends the overall boarding time, thereby actually making those with Elite status spend an overall longer period of time sitting on the plane.  Solution: Elite status is only used in consideration for upgrades and seat selection at check-in, not boarding order.
  2. Problem: Gate reps don't actually check the size of carry-ons.  In some cases, preemptively removing larger carry-ons and forcing gate-checks would dramatically reduce overall boarding time since the large bag causes an additive delay effect.  That is that it not only slows down the individual with the large bag, but also affects every passenger that boards after that passenger because a large bag taking up too much space forces "bin hunting".  Solution: Actually enforce the existing rules?  Seems too simple to be true.
  3. Problem: No one keeps their outerwear until the boarding process is over.  Nothing more frustrating than people stuffing their outerwear in the overhead before everyone has boarded.  This takes up valuable overhead bin space and, again, delays the boarding process for everyone that comes afterwards by forcing bin hunting. Solution: Add coat hooks on the seats and people will more likely just hook their outerwear on the back of the seat for easy access.
  4. Problem: Some people just don't give a damn what the gate rep calls -- they're just going to get onto the plane anyways. Solution: See below.
  5. Problem: Because of a combination of the above, it often causes folks to end up back-tracking and bin-hunting.  Sometimes, it also causes folks to preemptively place their carry-ons in forward rows, thinking that there is no space in the back rows, which only further exacerbates the situation. Solution: See below

The whole process is generally full of chaos, completely inefficient, and seems dated -- as if no one has stopped to think about these issues and how to solve them for decades.  So it's interesting to see this guy -- Dr. Steffen -- propose something new.  But it got me thinking that this guy's method still has an issue: it's not realistic for gate reps to call out this seating scheme (one row, one aisle at a time).  Yet this is what it demands as the process breaks down if you have passengers in row n-2 blocking passengers in row n.  Furthermore, the Steffen method makes an assumption about the nature of people: that they will obey order and think in the best interest of everybody.  When the gate rep calls the aisles for rows n, n-2, n-4, n-6, all of those folks are going to rush the counter at once in a single mass.

So I think an extension or more realistic approach to this method is necessary to get it to work: a new passenger queue system at the gate.  It should be -- like a plane -- one center aisle with several color-coded lines on each side of the aisle and just have people queue up in "bins" first.  It shouldn't match the rows in count, but should be at a 1:3 ratio per bin.  In essence, it's a modified block boarding pattern, but we break it down within the block so that, for example, window aisle passengers in the block are in bin "Purple", middle seat passengers are in bin "Blue", and aisle seat passengers are in bin "Green".

The gate rep would call bin "Purple 1" to board the window seat passengers in the rear of the plane.  Then call "Purple 2" to queue the passengers in the window seat in the middle of the plane, then call "Purple 3" to queue the passengers in the window seat near the front of the plane.  However, the real efficiency is in having the folks get into the bins first, before they are queued up.  The current Continental system results in 60 people all standing around, blocking the entrance of the queue when all that's needed is a bin system so they know where to line up.

This system acknowledges that the ideal boarding pattern cannot be achieved in real life and thus only attempts to gain efficiency by segmenting the users by zone in a pre-boarding line first.  Now everyone isn't standing around in one big mass, blocking the boarding line and rushing the line as soon as their block is called.  It's a hybrid of block and WILMA (which is what the Steffen method is) but not as granular as the Steffen method and adds an extra layer of organization at the gate to account for real-world considerations.


Lesson Learned on SharePoint Service Applications

Posted by Charles Chen

If you're setting out on writing your own SharePoint service applications, there is an important lesson that you should keep in mind (instead of learning it the hard way): ensure that all of your proxy, application proxy, service, service application, and service instance classes have public parameterless (default) constructors.

Otherwise, you'll have a heck of a time starting, instantiating, and uninstalling services with lots of MissingMethodExceptions and "{class} cannot be deserialized because it does not have a public default constructor" error messages.

Oddly enough, one thing I've learned from this is that the STSADM commands are often more "powerful" than the equivalent Powershell commands.  For example, Remove-SPSolution, even with the -force parameter, still failed with the aforementioned exceptions.  On the other hand, stsadm -o deletesolution {name} -override seemed to work fine.  Puzzling, for the moment, but it got the job done.  Similarly, stopping a service application that's AWOL (stuck on the processing screen) can be accomplished with stsadm -o provisionservice. Deleting it can be done using stsadm -o deleteconfigurationobject (though this one does seem to have side effects...).

Seems that Powershell is still a second class citizen when it comes to basic SharePoint command-line management.

But in any case, if you set out building your own service applications (<rant>and damn it Microsoft, can't you put some better examples out there?!  Even the few that are out there are convoluted, missing key details, hard to follow...</rant>), be sure to include public, default, parameterless constructors.

Filed under: .Net, Rants, SharePoint No Comments

Why Teachers Are _Really_ Leaving the Profession

Posted by Charles Chen

I caught a portion of an interview last night on NPR as I was in my truck, leaving Lowes.

I sat there for a moment, dumbfounded by what I was hearing and entirely outraged by the bullshit that Republican Wisconsin State Senator Glenn Grothman was spouting.

LYDEN: Last week, the Associated Press reported that nearly 5,000 Wisconsin teachers retired since the beginning of the year and that's more than half of the number from 2010. It's not a great way to start the year. Could the Republicans who passed this bill have done a better job of talking about it?

GROTHMAN: Well, I'm trying to talk about it right now. I will point out that at least one of the reasons why more teachers are retiring - we have heard anecdotal evidence that some of the worst teachers not waiting around for the inevitable and finally deciding it's time to retire. And I think some of those teachers who shouldn't have been there all along realized that without collective bargaining their time is up.

I was kind of riled up by this given that my wife is a former teacher (now an Learning Disabilities Teacher Consultant, but still works in the school system).  I've seen the dedication that she put into her work, I've seen the long hours grading papers and dealing with parents, I've seen results of her labor when parents stop her in stores years later and thank her for transforming her kids, I've seen parents specifically request to be placed into her class.

For all of this, she collects a relatively meager paycheck, but it's her passion and it's rewarding for her to be involved in the education of children -- in one capacity or another.  The low salary is made up for, somewhat, by the benefits that she receives.  Today, she came home with news about our insurance premiums: we're now required to pay about $280/mo. more for our health insurance plan because of changes enacted by Chris Christie and possibly the entire sum in a few years time.  To be clear, the sum of those premiums for a family of 3 is pretty much more than half of her monthly income -- enough to pay the mortgage on a small house.

I don't want to get into a long discussion on whether tax payers should or should not pay for the benefits of public employees, but at the end of the day, it is my belief that public employees are by-and-large hard working, middle class folk who provide valuable services to the taxpayers that are often hard to put a price tag on.  A good teacher can set a child on a path for success and help produce members of our society that contribute to our prosperity instead of being a drain on it.

But how can we possibly entice highly qualified candidates to these important jobs when as a society, we decide to strip these individuals of the benefits which largely make up for the otherwise less then illustrious labor of love?  Even those with a passion for teaching and helping transform kids will have to do a double take now and consider alternative career paths as we the movement against labor and the public sector continues to erode the middle class in America.

So I say to Glenn Grothman: the reason that teachers are retiring from the profession or quitting out-right is because society has created an environment where it's become untenable to put in the long hours and dedication required to be a good teacher for the already low salaries.  As the benefits that used to make up for the low wages continue to be eroded, the financial calculus will surely force more teachers -- especially the smartest and most qualified -- to reconsider their career choices for more profitable alternatives.

Post script: I think this is another case of Republicans driving the public sector to fail (by creating a dis-incentive for the most qualified, most skilled applicants) so that they can later point to the failure of the public sector!  Well, no shit, Holmes!  If you set the system up for failure, of course it'll fail.  Instead of the worst teachers leaving the profession, it's the best teachers that will leave the profession for better paying opportunities because these individuals are the ones that have the highest qualifications, the highest levels of professional ethics, and the highest ability to translate their skills outside of the classroom.

Filed under: Rants No Comments