We like success stories. Dished up well and accompanied by a slick Ted-talk they turn into best seller books, heaping success upon success. Tales like that make generous people feel good and jealous people feel miserable. Anyhow, I think we should read more about failure, especially in IT. It has much more to teach us.
Failure is everywhere and the longer you live the more failures you can chalk up. Half of all marriages fail. Most professional actors and musicians fail to make a decent living. I failed my driving test three times. Before his glorious revenge Steve Jobs failed pretty badly as a father and Apple founder first time round. The 1984 Macintosh was a commercial flop. In fact, most products fail. Clever people with a terrific track record can still fail spectacularly. Sometimes their road to failure is documented that we might learn from it.
Dreaming in Code (2007) by Scott Rosenberg is one of such books, and an all-time favourite of mine. Written for a non-technical audience Rosenberg follows the inception and first releases of the open-source Chandler project – not related to Friends, but named after the author Raymond Chandler.
Chandler was the love child of millionaire and Lotus 1-2-3 pioneer Mitch Kapor. With a team of ace developers he set out to develop a new personal information manager to handle email, reminders, calendars and notes. It was to be a radical re-imagining of concepts that would challenge Outlook and Exchange head-on. Unsurprisingly the team found out the hard way they had bitten off more than they could chew and could not deliver on their ambitious must-haves any time soon. If you want to bring about a paradigm shift better not commit to a shipping date. Just because you don’t like Bill Gates’s business model and curse Exchange doesn’t mean you can do a better job in half the time with a fraction – two dozen in fact – of Microsoft’s workforce, however noble your intentions. Illuminating is a quote from a 2004 interview with Linus Torvalds. He argues that if you set out with grand ambitions you may never get there: “Nobody should start to undertake a large project. You start with a small trivial project, and you should never expect it to get large. If you do, you’ll just overdesign and generally think it is more important than it likely is at that stage. Or, worse, you might be scared away by the sheer size of the work you envision”.
Chandler’s final release was in July 2009, two years after Dreaming in Code was published. It is well and truly dead by now. What’s great about the narrative of the book is that there is not a hint of schadenfreude for the poor developers’ enthusiasm, while bugs and other challenges were mounting. The book’s moto is Donald Knuth’s concise: Software is hard.
Of course, there have been more flagrant and much more expensive software cockups, but Rosenberg’s account is one of the more humbling and educational for fellow developers because many of the wrong turns taken were of the team’s own choosing. They had all experienced failure in the past, but they liked to think that this time it would all be different. We are all optimists and believe we are made for great things even though it is more prudent to start small, focus on optimal value over cool features, invite regular feedback and adjust your targes accordingly. This is what being agile is all about and it brings me to the second book.
The essence of Scrum is not about story points or mad/sad/glad retrospectives. Teams who only adopt the outward trapping of Scrum are in danger of becoming zombie Scrum teams. Another great and more recent book about team failures is Zombie Scrum Survival Guide (2020) by Christiaan Verwijs, Johannes Schartau and Barry Overeem. A playful and apt metaphor: it is scrum in-name-only. It has the meetings, the JIRA board, the planning poker, the sticky notes but none of the commitment or organizational mandate and so loses out on the benefits. It’s just an ineffective layer of ceremony wasting time, money and motivation. Dead on the inside.
A Scrum process can be defective because the team is just not interested in working closely with end-users and stakeholder out of a mistaken I’m-only-hired-to-code stance. More often though there are organizational barriers to establishing a close feedback loop and delivering a frequent release. Many teams are also not sufficiently empowered by the organization to be self-managing, which can lead to a feeling of apathy, reinforcing the impression that they are not equipped to be self-managing in the first place. Anyway, I cannot do full justice to the book in these few paragraphs. Do yourself a favour and buy it.
There are many useful takeaways to be had, but these are my favourite lessons learned:
An agile transformation is a radical change in the way you work, think and interact. It’s impossible to roll it out with an afternoon workshop and a copy of the Scrum guide. With some organizations it can take a long time and with some it just will not work.
Scrum is not about improving efficiency (building things faster, hence cheaper), but about effectiveness, about building the right thing. This is crucial ammunition when you have to counter the argument that these bi-weekly releases and review sessions with stakeholders are more expensive than a half-yearly release.
In a traditional, plan-based approach you carefully aim the bow and release the arrow. Success or failure becomes a question of hit or miss, as you cannot adjust the arrow mid-flight. Scrum is more like flying a plane, constantly adjusting the controls for to respond to the elements. I imagine flying is like that, but I would surely fail miserably as an airline pilot.