One of the lessons I've been mulling about the past few weeks is the importance of scope when delivering software.
Delivery of software can be thought of as a balancing act between three buckets:
- Time - this is the schedule and how much time you have to complete the work.
- Money - this includes everything from spending on more resources, better resources, tooling support, and so on
- Requirements - this defines what you're building
These are the three basic buckets that constrain the scope of what can be done and they react to each other in different ways. If there are more requirements or the requirements are inflexible, then it necessitates more time or money or both. If both requirements and time are inflexible, then more money will be required to achieve the goals within those limits. If money is constrained (less resources), then you must allow more time to deliver the requirements or trim the requirements.
Having been in consulting and in software development, each project has different priorities on which is more important and these priorities drive the sizing, cost, and pace of the project.
But in software development, I think one thing that I think many folks -- even experienced individuals -- get wrong is the fixation and inflexibility on requirements. I think that requirements are really much more fluid in software development projects as compared to contract-driven consulting projects. The reason is simple: in software development, the assumption is that there will always be another version and there will always be another point release; this is exactly what roadmaps are for.
Plan a solid roadmap and if you don't get a particular feature or capability in this release, you and your customers should have a good idea of which release it will be in down the road.
Some tend to lose sight of this and think that all features must be shipped in a constrained timeline. I think this is a losing proposition that forces a team to compromise on quality by artificially forcing requirements into a release when the reality is that there are often really critical features that must be delivered and there are nice to have features that would be great if they could be delivered. Teams and leaders without discipline and a focus on the criteria of success will have a difficult time discerning the two and lump nice-to-haves right alongside the critical work items. This is a recipe for failed projects, missed deadlines, and poor quality.
The reality is that software rarely comes out fully baked unless you're NASA launching a space mission worth billions of dollars and you only get one shot; most teams are not under such otherworldly constraints. There will always be something that could be better or easier to use or some missing feature discovered along the way or some new idea. The trick for teams that succeed is being about to create boundaries on what is needed now.
Apple shipped 7 versions of iOS before adding support for third party keyboards and NFC.
NPR's first mobile site was terrible (I don't have a screenshot of it, unfortunately), but at least they shipped it and their audience could use it and now they've evolved it to be a clean, responsive web site.
Here's what the an early version of Facebook looked like:
Microsoft shipped Azure without support for virtual machines until 2013.
But what if instead of new features, we're talking about bugs or design flaw? There is an important calculus at play here and I think that one way to think about it is like this: if there is a bug or design flaw that is preventing increased growth in userbase or revenue, then that takes precedent over any other bug or design flaw that is in the shipped system. Think about it this way: if you ship four versions of the software with the bug or design flaw, chances are, you can probably ship a fifth without addressing it (of course, this is not always the case, especially if the flaw is related to security). But if a bug or design flaw is stopping adoption or holding back revenue, then that flaw automatically becomes the most critical focus for a release.
The point is that in software product development, more often than not, the winning strategy isn't to get it perfect (as much as Steve Jobs would have you believe that he got it perfect each time, the fact that there was a next release meant that it was intrinsically not perfect -- there was always something to improve or add); it's to get it out and ship it and acknowledge that there will be a future release to add features or other improvements. This really allows the team to focus on what's critical now and get it out the door and on time.
To that end, roadmaps are important as a communication tool and a lever for controlling scope because it gives customers visibility and a sense of certainty that while feature X is missing in this release, in 3 months, it'll be in the next point release. It's important because it helps manage the requirements bucket; without a roadmap, the tendency of the team and customers -- in my observations -- will be to assume that every requirement is critical. It's a purely psychological notion because the lack of the roadmap makes it difficult to allow the team to offload some ideas and lesser requirements so that the team can focus on the requirements that are truly necessary to ship it. Without the concrete notion of The Next Release, the feeling will be that everything must be crammed into this release.
Ultimately, I think that for software development teams to successfully ship software -- given the typical constraints of time, money, and requirements -- it's important to be able to take a critical eye to the requirements and really be able to categorize and define the scope of what is critical versus what is nice to have. A clear roadmap is an important tool to help teams organize work and thoughts as well as communicate intent to customers.
From Peopleware, 3rd Edition:
The propensity to lead without being given the authority to do so is what, in organizations, distinguishes people that can innovate and break free of the constraints that limit their competitors. Innovation is all about leadership, and leadership is all about innovation. The rarity of one is the direct result of the rarity of the other. (p. 101)
Reflecting on the past year and a half, I've come to some conclusions on how development teams can be successful in delivering software. I don't think any of them are major or relevatory, but I think each team and each project has a different take on the same ideas. Here are mine:
During our active build phase, we release a new build every week and each build is functional software that incrementally incorporates requirements and defect fixes from previous builds. This has given us the benefit of allowing our customer to preview the software and help us identify flaws or areas requiring additional clarity in the requirements continuously and early in the process.
It might sound insane, but it is possible to release weekly builds because our solution incorporates a heavy dose of automation where it counts on many levels.
- We've removed raw SQL from the equation, relying purely on FluentNHibernate and NHibernate to automatically generate our schema
- We've invested in building tools to automate the export and re-import of configuration and data, allowing us to easily and quickly reset our development environments entirely with standard test data (bonus: the same tool allows us to move configuration and data from environment to environment)
- We've invested in idiot-proofing our installs so that they are boiled down to a few scripts
- We've built automated build scripts that package everything up neatly and even FTPs a build to our integration and test environments
- Our domain data model is 90% generated automatically from content schemas (SharePoint content types) which we have to create anyways.
Because of the automation, tasks which would otherwise be expensive are cheap to execute.
It also cuts down on mistakes and missed steps.
Our team is 100% geographically dispersed with developers and team members in Vietnam, Mexico, Virginia, New Jersey and California.
But relatively speaking, we meet very infrequently. Two development team meetings a week: one at the start of the week -- our "A" meeting -- and one towards the end of the week -- our "B" meeting. We use the "A" meeting to discuss our deliverables for the week and the "B" meeting to discuss the outcome of our weekly sprint walkthroughs, any adjustments that need to be made, and so on.
We also use these sessions as show-and-tell to let everyone see the progress and changes being made by different team members as well as to inform of upcoming breaking changes and mitigating actions required downstream.
Otherwise, developers are encouraged to have long spans of uninterrupted work time instead of constantly being pulled into meetings. One-on-one sessions and communications occur as necessary, but this recipe has been very successful in minimizing the amount of time the team spends huddled up in conference calls and gives everyone more time to solve problems.
Meet with a Purpose
Every meeting should have an agenda and an outcome (an action, decision, or an issue for followup). Demand a bullet-listed agenda when someone sends you a meeting request and provide one if you need to schedule a meeting. Ensure that the goal and outcome of the meeting is clear for all parties and schedule new meetings to resolve items not on the agenda or do not contribute to the outcome.
Additionally, create a record of every meeting. Who attended? What was covered? What was not covered? What was left open? What are the action items? Ensure that this record is easily accessible (wiki or forum system is perfect for recording these details) and email a copy to all participants and other relevant parties to ensure that everyone has the same understanding of what was just discussed. This basic task can often clear up misunderstandings before they become systemic issues. I take the burden on myself to record and followup with major bullet points from the meetings and it's saved my butt many times when following up with customers.
This is the simple art of not running a terrible meeting.
Lead by Example
A bit of career advice for those with a passion for software development: never remove yourself from the process of creation.
I have witnessed it as the career ladder moves individuals up and up, further from the pure act of creation that is software development. For those of us who feel invigorated when we solve a difficult programming task, for those of us who feel a great rush of exhilaration when a machinery of thousands of lines of code executes in harmony, it is our burden to tinker, to learn, and to create.
When you "ascend" from developer to architect or team lead or such, never leave your passion for creation behind; authority flows naturally from understanding, knowledge, and mastery -- not just a title.
I was inspired to reflect on this by an interview with James Dyson in Wired:
Wired: Now that Dyson is a sprawling, multinational corporation, how do you keep the spirit of innovation alive?
Dyson: We try to make the corporation like the garage. We don’t have technicians; our engineers and scientists actually go and build their own prototypes and test the rigs themselves. And the reason we do that—and I don’t force people to do that, by the way, they want to do it—is that when you’re building the prototype, you start to really understand how it’s made and what it might do and where its weaknesses might be. If you merely hand a drawing to somebody and say, “Would you make this, please?” and in two weeks he comes back with it and you hand it to someone else who does the test, you’re not experiencing it. You’re not understanding it. You’re not feeling it. Our engineers and scientists love doing that.
As a team lead, never just be a middle man with the developers and requirements; be an active participant in the process. Work on the hard problems. Understand the creation process and understand the challenges of the team from the bottom up and build your authority from your ability to innovate and solve problems.
If you watch shows like Iron Chef or Chopped, every one of the judges and every one of the Iron Chefs can be considered the vanguard of their craft and it is from there that their authority flows. You would not watch Iron Chef if all the Iron Chefs did was design the menu and then watch their team cook. You would not trust the judges on Chopped if they weren't great chefs in their own right that understood the ingredients, the techniques, and the skill required to pull off a dish.
The better you understand the system, the better you understand your team, the more effective you will be in understanding the root cause of an issue and how to route defects within the team.
Push Your Team Incrementally
As a young developer, I always found great satisfaction in solving new problems and new challenges. I think it's important that throughout the process, you push your team members and give them tasks that challenge their knowledge and abilities to push them just a little bit.
Of course, there will be plenty of mundane code and defect fixing, but don't box in your team members intellectually. Understand their capabilities individually and push them to try things that are just beyond their current level of capability, understanding, and comfort zone. This will keep them engaged and improve their skills to boot.
Invest in Code Quailty, Especially Early On
It's a lot easier to write good code earlier on in the process than it is to come in and try to refactor later on. Additionally, code written early on tends to be reused more often and patterns and solutions are copied by other developers later on. So early in the process, it is important to keep code quality in mind and correct bad behaviors since the most influential code will be written earlier in the process rather than later.
What this means is that detailed code reviews are far more important at the beginning than at any other time in the project. If you can correct bad programming practices or show a developer a better, more modular way of writing a piece of functionality early on, she will carry that knowledge throughout the project.
We rarely do code reviews now (1 year in) as I focused heavily on working one-on-one with developers as they joined the team to ensure that the code was up to standards. I frequently rejected code and asked developers to rewrite whole pieces of functionality to help them understand why one design was better than another.
Put Your Best Developers on Your Least "Visible" Code
What this boils down to is the framework level components. Your best developers should be working on the deepest innards of the system that power what the rest of the developers do at the presentation and business logic layers. This code will be the most invoked and the most reused so it is important that it is:
- Easy to use and as intuitive as the platform you are working on
- Well structured and object oriented to reduce repetition of code and code complexity
- Well documented with abundant examples -- your best developers must embody and practice whatever best practices have been designated
Do not waste your best developer's time with defect fixes (unless there is sufficient bandwidth), even if they can do it better than anyone else on the team because it will throw off the balance of the team (your more junior developers might not be able to fix a low level defect as quickly, but there are many design issues and higher priority defects that they cannot solve effectively yet).
Document, Document, Document
Early on in our process, I had to decide between a wiki system or a Word document for our documentation. Because of the fast, iterative nature of the project, I decided to use a wiki/forum system as it was more flexible and -- in a sense -- more "visible".
While our formal documentation is trailing, it is easy to assemble it from our weekly sprint guides which document every new feature with screenshots, details, and examples.
But at any given time, our customer and our delivery partner can load up the wiki and see exactly when we delivered a feature, how to install and configure the feature, how to use the feature, and so on. By putting it all out there in lock-step with the weekly delivery, it is easy to ensure that the entire team is aware of the status of the project and progress being made and allows test teams to progress iteratively so that by the end of the project, most features have been tested for weeks.
Mid ways through the project, we moved from "status" focused meetings to "demo" focused meetings where we would do a weekly writeup and walkthrough of what changed, what was added, and what was fixed. It also allowed for open forums for test and business teams to ask questions and get clarifications.
This approach has allows the customer to see progress and the customer will never be surprised at the end of the project as they will have seen the progress and documentation update on a weekly basis.
So far, we have done well with these basic guiding principles.
I'm sure I will revise and add to my own lessons learned as the project continues, but I think that this is a good starting point!
One of the things I've had to do quite often in the last few years is conduct technical interviews.
It's always a challenge as of course, if you want to make sure that a resource is technically competent in specific tasks or role for which you are resourcing, that's pretty easy; just ask questions oriented around those tasks that they will be responsible for. But what if you want to assess an individual's broader technical competency of and experience with a platform?
As a technically oriented guy myself, it's very easy to inject my bias and my core knowledge into the equation and I think that typically makes for a bad interview. There are always many things that I know well and many things that I don't know well for any given platform. So how can I go about measuring a candidate's competency without injecting my own knowledge and experience bias?
As an addendum, for technical interviews, it's sometimes difficult to come up with a good set of questions that are "fair". I don't think it's fair to ask obscure questions for which few folks would know off the top of their head, but would have no problem solving with Google and StackExchange, for example. I also don't like to ask brain-bender type questions as I don't find the outcome of those questions to be generally useful in evaluating technical expertise.
One approach is to ask open-ended design type questions. "Given this platform, how would you design a solution to meet requirement X?" "What are the benefits of approach U versus approach V for modeling this data?" These are okay, but I find that I often get clouded by situational bias. What I mean by that is that I tend to think of problems I've solved in the recent past or problems that I'm working on now. But I know that many of these design issues took me days if not weeks of research, prototyping, experimenting, and discussion to settle upon -- it simply doesn't seem fair to ask a candidate to produce a response on the spot. It's worth something, I guess, if they are able to come up with the same solution (or a better one!), but if they can't, does one hold that against them?
So in thinking about these issues, I think one good approach is to go with a reverse technical interview. What this means is that I ask the candidate to produce a list of technical and design questions for me for the interview. My thought is that this allows me to turn the bias that I would otherwise have into a tool because they will have the same biases. They will tend to ask questions around what they've worked on, what hard problems they've solved, and what experience they have. This seems like a much more dynamic approach and would seem to provide more valuable insight...I think. It's one thing to list a skill or a technology on your resume, but it's another thing to be able to ask deep, challenging, technical questions around it.
As a bonus, being able to come up with and ask good questions is itself a valuable skill.
Finally got around the cleaning up my office (after 4 years...) . Love the way it turned out and much more work space with much less clutter.
For reference, this is what it looked like previously.
From an NPR interview with Walter Isaacson, Jobs' biographer:
On Jobs' father, who rebuilt cars, and held design and craftsmanship in high regard:
"He would show Steve the curve of the designs and the interiors and the shapes ... and even have pictures of the cars he liked the most in the garage. He put a little workbench in the garage, and he said, 'Steve, this is now your workbench.'
"One of the most important things he taught Steve was [that] it's important to be a great craftsman, even for the parts unseen. When they were building a fence, he said, 'You have to make the back of the fence that people won't see look just as beautiful as the front, just like a great carpenter would make the back of a chest of drawers ... Even though others won't see it, you will know it's there, and that will make you more proud of your design.'"
I can't begin to compare myself to Jobs, but I think this is a value that's important to me, personally, from a software development perspective. There is an aesthetic and a beauty to well written code or framework not to mention the pride of the author in crafting a clear, concise, and elegant design -- even in the "parts unseen" by the end user.
I once got into a heated debate about code quality with a CEO who was, at the time, bent on fixing nail pops and seams on the walls of his recently built and painted house. I asked why he didn't see the same need to put an effort into addressing analogous issues in our code that disturbed the overall quality and aesthetics of the codebase, which seemed to me even more important than a few nail pops. "That's different" is the only response that I got, but to me, it's the same. There's a personal pride in building a product with exemplary craftsmanship. There's a team's pride to being able to walk into a code review with a customer and know that they will be wowed. There's a developer's own pride in writing software that takes design and usability of the framework into account.
This gets lost in the age of outsourcing and rent-a-coders, but great software is still -- and likely will always be -- a craft that requires skilled craftsmans to build (of course, great software requires more than that like vision and an understanding of the problem domain as well).
As a consultant, I feel strongly about giving sound technical advice to my clients, even if such advice means saying "no" to a client or possibly turning back a larger project for a more pragmatic one. It's about doing the right thing and offering sound technical advice to the best of my knowledge -- not just money, projects, and utilization.
The one personal example that really sticks out for me is the case where Microsoft sold a deal to a hedge fund to build a bulk import system using BizTalk that would have cost them triple the price (once licensing and hardware was factored in) of doing it using SQL Server DTS, which was easier to program, maintain, and more robust in every way (not to mention this company already had SQL Server skillsets in-house). Luckily, we were able to convince the client that DTS was purposefully designed for carrying out bulk import and transform of data before they committed the cash to BizTalk.
Recently, a friend of mine showed me a project that the Big Consulting Company he works for was delivering to their client, a public library. It looked really good for a public library website...until he dropped the bomb that it was built using Silverlight (and to top it off, he was really proud, too -- as if I was supposed to find it impressive). I don't think I've ever done a bigger facepalm in my life.
As I've stated in the past, I have a strong disdain for the misuse of Silverlight. There are certainly scenarios where it should be used for building web sites:
- Streaming media
- Scalable 2D vector graphics and animation
- 3D graphics and animation
- Interactive games
And that's it! Beyond that, if a company wants to use it in their intranet site, it doesn't concern me as much because the environment is more homogeneous and controlled in terms of having the platform to run the Silverlight applications; it's their headache going forward. Besides, if it's a private, multi-national company, then by all means; if they wish to waste their capital and resources, that's their choice.
However, it is a damn crime to recommend Silverlight to any client building basic web applications that are Internet facing, especially a public library financed by taxpayers. I mean, people should be fired and embarrassed for offering such terrible advice. To begin with, few non-Windows devices natively support Silverlight (and even folks on older Windows OSes can't natively run Silverlight apps). iPad? iPhones? Android phones? Linux based netbooks? As sales of traditional laptops and desktops decline, it's important to factor in the presence of these newer platforms when designing a publicly facing Internet site. I would think that this would be even more important for a public library.
Now, if the site were media focused -- like a YouTube -- perhaps it could be forgiven; after all, HTML5 is still a moving target and supported only by newer browser versions. But this is a public library website that was listing books...It's as bad as websites that still use Java (yes, Java without the "Script") for image galleries or raindrop effects. It's as bad as websites using Flash for menus and menu rollover animations.
I would be embarrassed to be a part of the company or the team that sold and implemented this deal. A fucking crime to the taxpayers of the township with me as the perpetrator; no better than stealing money from your neighbors. I couldn't live with myself for being so evil.
Now, he told me that the client insisted on Silverlight and that it was they who wanted it done in Silverlight. To me, that makes no difference. As a consultant, it's my duty to provide sound technical guidance to the best of my knowledge and ability. If there is a more compatible, cheaper, easier to maintain solution built on a platform with greater longevity that solves the same problem, I will recommend taking that route, even if it takes me out of the running. It's our job as consultants to consult and to offer sound technical advice.
For you see, the client may not know or care for the difference between Silverlight and HTML5 or jQuery based UIs. The client may be under the impression that a given UI or bit of functionality is only possible because of Silverlight if that's what they've been sold and demo'd. The client may not understand the alternative solutions as certainly, for a non-expert, the difference between two types of wood -- for example -- aren't perceivable. The client may be enamored with one buzzword or technology, but it is our duty and responsibility as consultants (and decent human beings) to tell the truth because I'd like to believe that when I ask a contractor to come to my house for a quote or get a diagnosis from an auto mechanic, he'd do the same for me and give me the low-down to the best of his or her ability and knowledge.
I'm still peeved by this as it's a critical misunderstanding of the Internet ecosystem and managing device compatibility as well as a critical misunderstanding of technology and their suitability for a purpose. Not to mention that it's a terrible choice for audience accessibility, long term costs, and maintenance. I really don't want to be upset by the fact that my friend or his team could have purposefully offered bad advice for greater financial returns as that would be a true embarrassment and I only hope that all sides in this come to their senses and ditch Silverlight.
In the end, for me, consultancy is about people and treating customers with respect by offering the best technical advice to one's knowledge. Even if it costs me my job, I've always believed that I am accountable to my clients and I'm responsible for giving sound technical advice.
Caught this editorial on CNN this weekend:
Companies spend billions on rent, offices, and office equipment so their employees will have a great place to work. However, when you ask people where they go when they really need to get something done, you'll rarely hear them say it's the office.
If you ask, you'll usually get one of three kinds of responses: A place, a moving object, or a time.
They'll say their house, their back porch, an extra bedroom they've converted into a home office, a library, the coffee shop down the street, the basement. Or they'll say their car, or a train, or a plane -- basically, during their commute. Or they'll say really early in the morning, really late at night, or on the weekend. In other words, when no one else is around to bother them.
Indeed, I think it's important to realize that different individuals have different productivity models. By that I mean that certain people are "morning people" and their brains are most active and creative in the morning. Others are "night people" where there brains are most wired and effective in the evenings. Some people feel more comfortable with natural lighting during the day time. Some prefer a bright working space while others prefer a dim one.
It seems counterproductive to force everyone into one model of the work environment when the preferences that maximize the efficiency of each individual can be vastly different.
And then there's the bigger issue of interruptions:
I don't blame people for not wanting to be at the office. I blame the office. The modern office has become an interruption factory. You can't get work done at work anymore.
People -- especially creative people -- need long stretches of uninterrupted time to get things done. Fifteen minutes isn't enough. Thirty minutes isn't enough. Even an hour isn't enough.
I believe sleep and work have a lot in common. I don't mean that you can sleep at work or you can work in your sleep. I mean sleep and work are phase-based activities. You don't just go to sleep or go to work -- you go towards sleep and towards work.
You aren't sleeping when your head hits the pillow. You start the sleep process. You have to go through phases to get to the really beneficial sleep. And if you're interrupted before you get there, you have to start over.
The same is true for work. You don't just sit down at your desk and begin working effectively. You have to get into a groove. You go towards good work. It takes some time to settle in, clear your head, and focus on what you need to do.
A very good analogy and I wholeheartedly agree. At the same time, to ensure that this model works, teams need the right tools (Webex or equivalent, chat clients, VOIP, etc.) and the right people to make it work. To some extent, it takes a good amount of trust that each member of the team understands their tasks and roles to get their jobs done without having to have a manager or supervisor constantly buggering for a status or having meetings to figure out the status of the tasks.
At least for myself, I find it incredibly difficult to work any any problem of moderate complexity without sitting down and having a solid bloc of a few hours to work on the problem. There's nothing worse than having to do a mental context switch when one is working on a difficult problem. Well, it's only worse when that context switch is for a meeting that's inconsequential to the tasks at hand
I didn't go, but John Peterson did.
Check out his feedback from the conference.
The session is titled "Object Oriented Development and Practices in SharePoint":
Building maintainable solutions on the SharePoint platform can be a challenge (and that might be putting it mildly). Code interspersed with CAML strings, rampant code duplication, hundred (thousand?) line methods, inconsistent code quality, and so on. How can a dev/technical lead address these problems that arise when a team of individuals with diverse experience and skill levels embarks on designing and building a solution on the SharePoint platform?
This session introduces a series of practices, tools, libraries, and techniques to support an object-oriented approach to building sustainable and maintainable solutions on the SharePoint platform. It offers an innovative approach to solving complex solution and development problems through embracing simplicity and leveraging the capabilities of the .NET Framework to build a framework for highly object-oriented, patterns based solutions.
Technologies: SharePoint 2007, Visual Studio 2010, C#, .NET, XSLT (Saxon)
Audience: SharePoint developers, SharePoint technical architects, SharePoint technical leads, .NET developers
Level: Intermediate/Advanced. Audiences with experience in design patterns, reflection, delegates, anonymous functions, and XSLT will be able to follow along and extract the most value from this session.
To expand on that, the plan is to cover some of the lessons I've learned from being deep in the code on a handful of large SharePoint projects. These lessons I've encapsulated in a framework of sorts which was designed to help:
- Accelerate development of solutions for SharePoint
- Increase developer productivity while still maintaining high levels of code consistency
- Increase adherence to the DRY (Don't Repeat Yourself) principle by leveraging patterns and object-oriented code
- Decrease the entry barrier for ASP.NET developers transitioning to SharePoint
It won't be for everyone; however, for any team that's deep into the SharePoint APIs and building custom solutions (web parts, event receivers, web pages, layout pages, and so on), I promise this will be a great session to attend. My hope is that attendees will be able to walk away with some ideas on how to make their teams more productive and to help teams write better code.
The event will take place on Saturday, October 9th at the DeVry campus in Fort Washington, PA (great campus, good presenters, free lunch!). Details here: http://codecamp.phillydotnet.org/2010-2/default.aspx
I'd be lying if I said I wasn't a bit anxious over the whole thing.
I plan on putting together a monster post before the event with the outline, details, and materials of the stuff I plan to cover. See you there (and wish me luck)!