This week marks the publication of memoir by Joe Ricketts, The Harder You Work, The Luckier You Get. I am the individual that led Ameritrade to the internet and spent many hours being interviewed for this book.
Like many software development teams in the 90s, we were struggling to keep up, as the internet – rather than floppy disks - became the data delivery method of choice and previously successful development models began to break down. This new paradigm was powerful because we could just update all our clients software at the flip of a switch. But it was also fraught with danger as it became very inexpensive and easy to launch new code without appropriate quality control.
The Egyptians were not Agile
In the olden days, almost all projects were delivered following a waterfall project management process. It is likely that even the pyramids started with some sort of primitive blueprint which included every thought and idea that the hoping-to-be-immortalized king might have in his head. As challenges to design crept-up during construction, they were dealt with via design compromises. As a result, end projects were almost always something less than they were initially intended – even if nonetheless cool.
As people got better at building skyscrapers, ships, and bridges, they got better at estimating the time a project would take. Estimates were created based on the average of past projects scaled up or down to match the one at hand. We can imagine the thinking of an engineer in 1890:
That last bridge took 2 years and was a mile long. This new one is 20% longer so it should take about 29 months.
As our society of builders approached the middle of the 20th century, waterfall project management was very nearly perfected for capital projects. Engineers were aware of the details, pitfalls, and requirements of their projects. In fact, they were good enough that we were able to put a man on the moon!
Old practices didn’t fit new products
But around the same time of Apollo 11, a new generation was coming of age who would challenge the time-tested results of waterfall with respect to software development. Capital projects had been physical resource based. Metal, factory space, rivets, steel, and labor had gone into building the monuments of the previous two millennia. The new monuments were to be made of information, communication protocol, mental horsepower, and electricity. They deserved a new method of project management that matched their intangible materials.
Taking cues from Japanese manufacturing, their own experiences, and the changing relationship between programmers and users, computer engineers began to advocate for smaller production cycles that were more flexible. They sought to replace the heavy process of waterfall with something lighter. They promised to increase their accountability and transparency in return for a seat at the planning table and the opportunity to honestly manage executive expectations. Enter Agile.
There is a romantic notion within the software development community that in 2001, a group of 17 computer scientists got together in Utah and invented Agile software development. But that is really not what happened. In truth, engineers all over the world had been wrestling with the problems of building software in executive-driven environments and by the 1990s were coming up with similar solutions. The birth of the internet, and the explosion of users demands, really kicked the discussion into high gear.
I was in Omaha Nebraska in the 1990s. Omaha might not have much of a reputation, but due to its central proximity and favorable demographics, it was an early hub for military, communications, and technology providers. My company was Ameritrade even though it was called TransTerra at the time. Our technology team was part of the greater Omaha tech community. We knew each other, met regularly to discuss issues and share a beer, and traveled to the same conferences. You can always spot a group of travelers from Nebraska because half of them will be wearing red or have the state name emblazoned across their body somewhere. It helps when they get lost.
As TransTerra changed its name to Ameritrade and its number of clients grew from thousands to millions, we were under unprecedented pressure to revamp the way we built software. At the time Amazon first bragged about $1 million days, we were already seeing $100 million days. Our development cycles were built around exploding demand, for which our capacity increasing releases were regularly inadequate. And every day we were learning how better to present our interfaces. Clearly, long, waterfall development cycles were not going to work anymore.
In the spring of 1998, following the crush of few 80 hour/week development cycles, we put a stop to the treadmill, and took a step backward. For nearly three weeks, the development team and I locked the doors, sat together, and wrote what was to be Ameritrade’s new software development process. We called it the Cooperative Software Design Process.
I wish I still had a copy of that original document, but it is long gone. What I do have is a PowerPoint simplification, dated a year later (1999), that was presented to executives and new management as part of an initiative I worked on with my buddy, Ronny Gal from Boston Consulting Group (BCG). It is interesting to review and note how many similarities there are to what we now all know as Agile.
I have attached one particularly interesting slide from that deck. Much of it will feel familiar.
We missed a lot of important Agile components too. We didn’t think of time boxing, completed software instead of status reports, or daily stand-ups, but those things probably weren’t appropriate to Ameritrade at the time. We were creating a process that allowed our business units – and subsequently the end users – to understand that they were in control of what we were doing. In retrospect, it was quite appropriate.
The emergence of the internet changed a lot of things in the mid-90s. Companies that didn’t embrace it went away and those that did were forced to re-evaluate the way they did business. Few companies were as shaken and shaped as much as Ameritrade. Every element of the company was put into flux – but nowhere greater than the what became the internet development team. We were pioneers out of necessity and lucky that the pre-Agile discussion was one of which we could be a part. These concepts helped our team progress to maturity and paved the road for Ameritrade’s growth and eventual position as one of the largest brokerages in the country.