|
Scrum: software product development with agility
Pete Deemer,
Chief Product Officer,
Yahoo India R&D
The traditional way to build software, used by companies large and small, is commonly known as ‘The Waterfall.’ There are many variants, but it typically begins with a detailed planning phase, where the end product is carefully thought through, designed, and documented in great detail. The tasks necessary to execute the design are determined, and the work is planned using tools like Gantt charts and programs like Microsoft Project. The team arrives at an estimate of how long the project will take by adding up detailed estimates of the individual steps involved. Once stakeholders have thoroughly reviewed the plan and provided their approvals, the team starts to build. Team members complete their specialized portion of the work, and then hand it off to others in production-line fashion. Once the work is completed, it is delivered to a quality assurance organization which completes testing prior to the product reaching the customer. Throughout the process, strict controls are placed on deviations from the plan to ensure that what is produced is actually what was designed.
This approach has strengths and weaknesses. Its great strength is that it is supremely logical: think before you build, write it all down, follow a plan, and keep everything as organized as possible. It has just one great weakness: humans are involved, and humans don’t work very well this way.
For example, this approach requires that the good ideas all come at the beginning of the development cycle, where they can be incorporated into the plan. But as we all know, good ideas appear spontaneously throughout the process—in the beginning, in the middle, and sometimes even on the day before launch, so a process that doesn’t permit change will stifle this innovation. With the Waterfall approach, a great idea late in the development cycle is not a gift, it’s a threat.
The Waterfall approach also places a great emphasis on writing things down as a primary method for communicating critical information. The very reasonable assumption is that if I can write down on paper as much as possible of what’s in my head, it will more reliably make it into the head of everyone else on the team; plus, if it’s on paper, there is tangible proof that I’ve done my job. The reality though is that most of the time these highly detailed 50-page requirement documents just don’t get read. And that’s probably just as well, because when they do get read, the misunderstandings are often compounded. A written document is an incomplete abstraction of a picture I have in my head; when you read that document, you create yet another abstraction, which is now two steps away from what I’m really thinking of. It should come as no surprise that serious misunderstandings occur.
Something else that happens when you have humans involved is the ‘hands-on aha’ moment—the first time that you actually use the working product, and you immediately think of 20 ways you could have made it better. Unfortunately, these very valuable insights often come at the end of the development cycle, when changes are most difficult and disruptive—in other words, when doing the right thing is most expensive.
Humans also have a poor ability to predict the future. For example, the competition makes an announcement that wasn’t expected. Unanticipated technical problems crop up that force a change in direction. Further, people tend to be particularly bad at planning things far into the future; guessing today how you’ll be spending your week eight months from now is something of a fallacy, and it’s been the downfall of many a Gantt chart.
The Waterfall also tends to foster an adversarial relationship between the team-members that are handing work off from one to the next. “She’s changing her mind about what she wants.” And this gets us to another observation about the Waterfall—it’s not that much fun to work within. We’d go a step further and say that the Waterfall is a cause of misery for the people who build products, and the resulting products fall short of expressing the creativity, skill and passion of their creators.
Many users of the Waterfall experience these shortcomings again and again, but it seems like such a logical approach that the natural reaction is to turn the blame inward: “If only we did it better, it would work.” If we just planned more, documented more, resisted change more, everything would work smoothly. Unfortunately, many teams find just the opposite: the harder they try, the worse it gets.
The Agile family of development methodologies was born out of a belief that an approach more grounded in human reality would yield better results. Agile emphasizes building working software that people can get hands-on with quickly, versus spending a lot of time writing specifications upfront. Agile focuses on small, cross-functional teams empowered to make decisions, versus big hierarchies and compartmentalization by function. Agile focuses on rapid iteration, with as much customer input along the way as possible. Often when folks learn about Agile, there’s a glimmer of recognition: it sounds a lot like back in the start-up days when we ‘just did it.’
One of the fastest-growing Agile methods is Scrum. Many teams using Scrum report significant improvements, and in some cases, complete transformation, in both productivity and morale. For product developers, many of whom have been burned by the management fad of the month club, this is significant.
Scrum is simple, powerful, and rooted in common sense. It organizes product development in cycles of work called Sprints, which are typically 1-4 weeks in length, and which take place one after the other without interruption. The Sprints are of fixed duration; they end on a specific date whether the work has been completed or not, and are never extended. At the beginning of each Sprint, a cross-functional team selects items from a prioritized list of requirements, and commits to complete them by the end of the Sprint; during the Sprint, the deliverable does not change. Each work day, the team gathers briefly to report to each other on progress, and update simple visuals that orient them to the work remaining. At the end of the Sprint, the team demonstrates what they have built, and gets feedback which can then be incorporated in the next Sprint. Scrum mandates that the team produce a product at the end of the Sprint which is really ‘done.’
Scrum is not a process. It’s a framework which enables the team and stakeholders to ‘inspect and adapt,’ offering flexibility and adaptability balanced by the stability required to produce demonstrable results. Scrum works by simply making visible the dysfunction and impediments that are impacting the team’s effectiveness so that the team and organization can address them. For example, most teams are not good at estimating how much they can get done in a certain period, and so will fail to deliver what they committed to in the first Sprint. To the team, this feels like failure. But in reality, this experience is the necessary first step toward becoming more realistic and thoughtful about their commitments, and also being even more committed to delivering what they signed up for. This pattern, of Scrum helping make visible dysfunction, of enabling the team to do something about it, is the basic mechanism that produces the most significant benefits for teams using Scrum.
One common mistake teams make, when presented with a Scrum practice that challenges them, is to change the practice, not change themselves. For example, teams that have trouble delivering on their Sprint commitment might decide to make the Sprint duration extendable so they never run out of time, and in the process, ensure they never have to learn how to do a better job of estimating and managing their time. In this way, teams can morph Scrum into just a mirror image of their own weaknesses and dysfunction, and undermine the real benefit that Scrum offers: making visible the good and the bad, and giving the team the choice of elevating itself to a higher level.
|