5 Cooks

I recently finished up Eric Brechner’s I.M. Wright’s Hard Code.

One of the more interesting aspects of development and project management that he brings up is the concept of working depth first as opposed to breadth first.  Too often, I think management gets this crazy idea in their head that progress is best served by having all hands on code at the same time; that to make the best progress, we should all be tapping away at our keyboards and churning code.  I think this mindset is a mistake, especially in small teams.

In his October 1, 2004 article titled “Lean: More than good pastrami”, Brechner (as I.M. Wright) writes:

Go deep

Of course, you can use Scrum and XP poorly by making the customer wait for value while you work on “infrastructure.” There is a fundamental premise behind quick iterations built around regular customer feedback: develop the code depth first, not breadth first.

Breadth first in the extreme means spec every feature, then design every feature, then code every feature, and then test every feature. Depth first in the extreme means spec, design, code, and test one feature completely, and then when you are done move on to the next feature. Naturally, neither extreme is good, but depth first is fa r better. For most teams, you want to do a high-level breadth design and then quickly switch into depth-first, low-level design and implementation.

This is just what Microsoft Office is doing with feature crews. First, teams plan what features they need and how the features go together. Then folks break up into small multidiscipline teams that focus on a single spec at a time, from start to finish. The result is a much faster delivery of fully implemented and stable value to demonstrate for customers.

To me, the key sentence is the last sentence: as software developers, project managers, and delivery teams, our goal should be to deliver demonstratable value to the customer in the fastest manner possible while maintaining a sufficient level of quality.

In small teams which work breadth first from start to finish, it becomes more difficult to accomplish this.  (Well, I guess this is true for large teams, too.  But in a larger team, you have more opportunities to modularize large stacks of the application.)

One of the core problems with working breadth first is that it assumes that everyone is a developer and everyone is equally skilled at every type of development task.  This forces developers into roles and tasks which they are perhaps not comfortable with, not proficient with, or perhaps not even very good at.  This may be a good thing, in the general case to act as a driver for learning, but at the same time, it’s not very conducive to delivering quality (especially on production code).

In a sense, this approach assumes that everyone in the kitchen is a chef when this may not be the optimal usage of the resources at hand.  No kitchen is comprised of all chefs; there is a head chef, a few sous chefs, a pastry/dessert chef, there are guys working on prepping ingredients, resources prepping the plates and dishes, resources optimizing the orders as they come from waiters, there are head chefs who are not working but experimenting or learning new techniques, and so on.  The point is that too often, management assumes that everyone in the kitchen is a chef and everyone in the kitchen should be cooking.  The reality is that no kitchen runs that way just as no development team can be run that way.   Fred Brooks captured this in The Mythical Man Month with the concept of a Surgical Team.

For the sake of efficiency and quality, it seems that product development would be better served by using the proper resource for the development task at hand.  Certainly, this raises the issue of pigeonholing developers into certain roles, but that’s what downtime is for: cross training and developer education.

The second problem with working breadth first is code and functional duplication.  In a multi-tiered architecture, if everyone is working on every tier, it becomes increasingly likely that certain functionality will end up being duplicated simply due to lack of immediate visibility.  In a depth first approach, a team might be responsible for writing the service interface; they will know intimately which services already exist and which services can be reutilized.  In a breadth first approach, in any non-trivial code base, functional duplication becomes rampant without a lot of non-productive effort.

The third major issue I have observed with a breadth first approach is that it causes stress to the test and quality assurance teams.  Instead of having testable features trickle in as they are finished, pieces tend to all end up in their queue in a giant tidal wave.  This puts more strain on small test and QA teams as it means that they are forced to work in a breadth first manner as well.  Instead of having multiple eyes running one test script to ensure that the component is compliant with the design requirements, you end up with testers rushing through test scripts by themselves trying to catch up.  Would it not be more desirable to have features delivered in a linear fashion to the test and QA teams so that their work can be more rigorous and comprehensive?  I think this helps improve quality by finding usability issues and bugs earlier on in the development cycle as opposed to finding all the bugs near the end of the formal testing phase.

Just as you wouldn’t deliver appetizers, the main course, and dessert to the table at once, it doesn’t make sense to drop every module on your test and QA team all at once; it makes more sense to hand them deliverables early and often so that their work and effectiveness is not constrained.  This is beneficial to dev as well since bugs and usability issues can be returned to the team earlier on in the cycle.  Of course, the beauty of working depth first is that, if you do not have the option of pushing back a release date, it’s easier to still ship on time with completed and tested modules minus features which could not be completed on time in the event that test or QA invalidates some earlier design assumptions which require significant time to fix or reimplement.  In other words, a depth first approach gives you the flexibility to deliver finished and tested code even if some features must be left out for a later release.

The fourth major issue is exactly as Brechner writes: by working breadth first, you are not delivering value to the business users or your customers.  Features are never in a fully tested and qualified state until the very end of the development cycle.  Using Brechner’s suggestion to design breadth first and implement depth first, it is possible to move completed pieces through test and QA (and fix bugs which may return) and deliver a working, functional module to business users or to customers, even if the particular product milestone is not complete.  This decreases the feedback cycle and, again, allows usability and functional issues to be caught earlier in a smaller number of cases, rather than later in one huge bucket of tickets.

The fifth major problem with a breadth first approach is that you spread your resources thin.  This means that you may not have sufficient resources who have knowledge of the code for a particular component.  This can be mitigated to some degree if you run tight code reviews where group A peers into group B’s code on a regular basis and has an understanding or the codebase or if group B generates impeccable documentation, but more than likely, you end up with small silos of knowledge where a small number of developers hold most of the knowledge regarding a module.  This is bad for any number of reasons, as you can imagine.

So next time management peeks its head into the kitchen asking why everyone isn’t in the act of cooking, perhaps you can sit them down and have a little talk with them; I think there is a lot of value to be found in Brechner’s suggestion to perform high-level feature design breadth first, but low-level design and implementation depth first.