An Invariant is a motto,a guiding light that–no matter where you’ve found yourself in the agile to waterfall spectrum, and no matter the rituals your product team follows–will help you ship quality software consistently.
Opening Acknowledgement: While I am the person authoring and posting this series on software development, these are not solely my ideas. Notably there are several people from Stack Overflow, Unchained, and Zaprite that have helped me form these opinions on how to build software. Some of them even named things I mention here as we developed our own identity as respective product and engineering teams. Those people are: Joel Spolsky, David Fullerton, Matt McManus, Parker Lewis, and John Magill. Notably, the atomic unit and invariant framing was developed by Joel, David, and myself as we battled to scale the product and engineering org at Stack Overflow, which I later built upon with Matt McManus at Unchained. The Invariant framing was a collaboration with Joel Spolsky internally at Stack Overflow, but I believe those initial internal writings are long lost, and this represents my own take on that framing.
If you’ve ever been on a team building software (product, design, engineering) or tangentially related (marketing, sales, ops), you’ve likely wondered if you’re doing it all wrong. Things take longer than expected, the end product isn’t what customers wanted, and getting to the end required Herculean efforts that are not repeatable. Because someone involved read a book one time, you get together a week or two after a project ships and do something called a retrospective, where no one blames anyone, and then you do the same damn thing all over again. If this is you, I want to propose an alternative way to think about software development.
Perhaps greater than the holy wars of linux vs windows, tabs vs spaces, emacs vs vim is ‘how the hell am I supposed to get seven people to agree and ship this damned product’ vs ‘holy shit I have to get 78 people to agree and ship this product'. The latter holy war has high priests known as Scrum Masters, Product Managers, and Product Owners. They have read the Good Book of the One True Product Development Methodology, and as long as you don’t sit during a meeting, you’ll ship on time. Or you won’t. You probably won’t, because methodologies don’t really matter.
I am concerned with the state of product development. Planning has either been demonized (you can’t ever know anything so just write code) or trivialized (our planning is a to-do list of tasks and features on a kanban board) which cripples the abilities of software teams to consistently do their jobs well. Waterfall, or anything that smells like waterfall, is grounds for mutiny. In this world where everyone is agile and we move fast and break things, we’ve handicapped the engineers, designers, and PMs who make software work, and we’ve given a pass to CTO’s and VP’s of Product who are supposed to, well, know what to do.
That’s why the companies I’ve worked with invented something that was not a methodology, but a process that can fit differing methodologies. If you’re on a product team shipping features, you may use Scrum or Shape Up. If you are on an SRE or DevOps team, a simple Kanban methodology may serve your purposes just fine. Once you’re at the point of defining how you and your team build software, your only actual limit is the capabilities of the people in that team. Nearly any methodology could work if followed by hard working competent people. I argue there are things good software teams do–I call these Invariants–no matter what methodology is employed, what title the guy leading the meeting has, or to whom a software team answers.
An Invariant is a motto,a guiding light that–no matter where you’ve found yourself in the agile to waterfall spectrum, and no matter the rituals your product team follows–will help you ship quality software consistently. The Invariants themselves have their own proto atomic units, which are the primordial soup by which Invariants are built.
All high functioning product teams, regardless of org charts and specific methodology do some form of these five things in their product development process:
Differently sized teams and different products will have slight variations, but there’s no getting around the atomic units of product development. The key to understanding the atomic units, and therefore the Invariants, is that we try very hard to solve problems at higher levels of abstraction before we move to problems at lower levels of abstraction.
No, it’s not waterfall development. We’re salmon, we can swim upstream.
If the strategy is wrong and we’re already writing code, we’ll fall back to strategy. If we’re ready to ship but have a design issue, we’ll fall back to discovery. We expect this to happen sometimes, because you inevitably discover things later on.
This is the core argument between the agile and waterfall camps. The agilistas pervert the agile manifesto and claim you can never know anything so why even bother writing stuff down and that changing code is actually fun and easy! While the waterfallistas claim that everything will be known in advance, nothing ever changes, and programmers are people that translate english words line by line into code that computers understand, and also what is a customer?
Both arguments, even when told in less absurd ways than I’ve chosen, are entirely unconvincing. As salmon, we can navigate both ways through phases, while trying our best to solve problems at higher levels of abstraction before solving lower level problems. We can also prioritize getting working software into customers hands as quickly as possible and iterate on their feedback as quickly as possible. We’re agile, we just Think Before We Do.
Hahahahahaha. No, almost no teams do this, at least not consistently over a long period of time. If you’re skeptical, do three things for me real quick:
You’d be lucky in most organizations to get positive results on one or two of these challenges. If you check all the boxes, give your product and engineering managers raises. They are special.
The reason people don’t do these is that they form their org charts and their product development processes on a tactical basis that unintentionally results in a lack of planning and strategy. Product and engineering teams have been hoodwinked into following tactical guidelines that have us reinventing to-do checklists and calling it strategy (I do love checklists). Worst of all, we avoid accountability when everything is tactical. This in turn makes the actual building of software a nightmare, where code is constantly being changed and no one can remember why features are being built and for whom they’re being built.
Failure at a tactical level is supposed to happen and doesn’t doom a team or project. As Peter Thiel states, failure at a strategic or systemic level is a tragedy, and no one gets better from it. If your team’s process does not have strategic ownership that is owned by known individuals, true accountability is impossible to achieve. Now we’re venturing into invariant territory, so let’s get to it.
If the atomic units are the tactical descriptions of the things all good software teams do, the invariants are the strategic stories of why and how these tactics work. Each atomic unit has a corresponding invariant that we’ll discuss at length through this series. The invariants are:
These brief descriptions of each invariant are not enough to understand this process though. You’re going to have to bear with me and read this five-part series where we break down each invariant in as much detail as I can provide. I promise that if you stick with me, it won’t matter if you’re sitting or standing during meetings, you’ll have a better chance of consistently shipping quality software that matters.
Originally published in The Tools of Ignorance
