The Story
I want to share a true story of a relatively newly formed agile team. We were made up of ten engineers writing and testing code in C++, Java, and fully embracing the emerging javascript frameworks. The application we were building was both complex and complicated. We were learning how to launch a product, trying to embrace user-centric design, and honestly, I often screwed up as a lean agile leader. All that said, we were a pretty great team and we were constantly learning.
We were in the midst of a new product release that was careening towards a GA date. The testers on our team — who were pretty great at what they did — were getting pressed against that GA date with the build-up of feature inventory. As a team, we were doing an okay job at establishing automation, but we could do more — a lot more. So with the testers in the needs of some aid and the need to shift left more testing, we arrived at Testivus for the Rest of Us.
How It Started
As a team, we often themed our sprints. Sometimes these sprint themes were merely broad brushed sprint goals. We already had the “Unleash the fury, Mitch” sprint in which we literally sprinted to the finish line several key product features. In need of raising our game around test automation and in general, switch the team from a “test right” to “test left” where quality was truly the starting point of all work — we decided to theme a sprint “Testivus.” What better theme than Testivus — time to air our grievances and demonstrate feats of strength.
NOTE: In case you’re not a Seinfeld fan, Testivus is a play on Festivus. See Festivus – Wikipedia for more information.
In preparation, we built the sprint backlog around those items that needed the most attention (we looked at a heat-map of components based on bugs previously found, code complexity, and skeletal impact to the product). This backlog wasn’t just a bunch of technical stories, we focused on the functional areas and created functional validation stories that restated the user outcomes, and we refined the acceptance criteria based on usage validation (as well as early learning from the field – e.g., performance needs, essential data needs, etc.). Some may call this just a regression backlog, and that’s fair; however, our mindset was different — it was all about outcome validation and, to some extent, validation that the product we were making was innovative and would kick-ass in the minds of our prospective users. Also, to note, this wasn’t about a “Hardening” sprint. You could argue this; however, we weren’t at the end of the release, and our core goal was to bring testing left in the cycle.
Once we built the stories and the resulting tasks included fleshing out a test harness, more in-depth unit testing practices and practicing TDD, functional UI test automation, and refining our CI process to extend into CD including bootstrapping test and demo data creation.
The Results of the First Testivus
The tangible outcomes were:
- Stronger installation and deployment automation including packaging.
- A robust test data generator and demo data bootstrap.
- A new & improved test harness that handled both functional aspects as well as continuous performance benchmark validation. The harness covered over 1,800 scenarios nightly.
- Increased code coverage from single digits to a substantial double-digit percentage.
The most impactful outcome was that team’s mindset which was already pretty good, became hyper-focused on how quality should be built-in. From this point forward, during refinement, the first thing we would talk about is attributes of testing.
The Longer Term
Testivus became a regular occurrence in the context that we called for a “Testivus” day or two every 5-6 sprints. Sometimes the need surfaced when we noticed a trend of bugs in a particular area, field feedback, and a couple of times we “Testivus” as part of a quality hack day.
Reflecting on Testivus, there was a need and the theme was awesome. It changed our mindsets, we had some focused energy on quality, and we had some fun — even though it was intense at times.
The great thing is — this concept wasn’t unique to us. Sometimes teams need a cause and a theme to try something new, and this helped us gel around some learning of good practices.
A couple of years later I joined VersionOne where the development team adopted “meatfest” which was a bug bashing event that had the team grilling out meat and vegetables, and drinking some beer throughout the day — all the while just fixing a bunch of stuff (I would point you to the awesome blog on this topic, but someone took it down).
Please let me know if you have experienced the same or would like to discuss further. Thanks for reading!
Update: We definitely weren’t the only one’s going down the Testivus path — just Google “Testivus.”
