Often, organizations we work with are focused on output. It’s easy to understand and measure, and government leaders often use it as a metric to determine if a legal or regulatory requirement has been met. There is value in ensuring teams meet their contractual obligations, but an output-focused culture can create perverse incentives that can harm the outcomes products enable for users.
Output and productivity metrics became standard measures of success during the industrial revolution, which was all about the creation of goods like textiles and I-beams. It also created an imperative to keep the machines running, as that translated into more goods produced.
However, software and digital services are never “done.” We’re not building finalized products that are then put in boxes, shrink wrapped, and shipped. We’re building ever-evolving systems, and we have the opportunity to gather immediate feedback as we build. This enables an outcomes mindset in which our aim is to improve outcomes for users, not to produce more products or features.
Outcomes are admittedly harder to quantify than outputs, but measuring them can help ensure that not only are requirements met, but users’ needs are addressed. To get to an outcome-focused team or organization, you need to set the right internal incentives. But first, let’s talk about the potential harm of focusing primarily on outputs.
The costs of output-focused incentives
When teams are incentivized to deliver outputs, outcomes can actually be impeded. One of our teams worked on a healthcare application where there was a financial bonus for delivering all of the planned features in a given period. Through user research, the team found that one of the planned features was not creating any value for end users, for the agency, or for any stakeholders who supported the application. Not only was it failing to reduce the burden to end users, lower support costs, or provide actionable information to the organization, it was also going to be a significant effort to develop.
The planned feature was to split an end-user workflow into two discrete steps, but the research revealed that there was no need to do this; no one had a need to do the first step without completing the second as well. In fact, the team learned that splitting it into two steps would lead to confusion if the first step was done but the second wasn’t.
When the team realized this planned feature wasn’t solving any user problems or providing business value, they thought it would be a success to drop the feature in order to enable them to deliver other valuable work and save time and money. However, since the contract incentivized the product team to deliver this feature, it required several difficult negotiations to scrap the feature and resulted in a few weeks of churn, due to that focus on output. If we had gone forward with the feature, outcomes would have actually been harmed.
Sometimes a misguided attempt to optimize outputs can backfire, even resulting in reduced output. We’ve seen some examples where a feature in progress is shifted from one delivery team to another, based on capacity. A process framework that many of our programs use instructs that any team should be able to pick up any user story with no ramp up. The theory is that any idle capacity is wasted, so there is a desire to hand in-progress work to whichever team has capacity to take it on, based on a person joining or leaving the team or someone going on vacation.
In practice, however, products are often complicated and broad, and teams have to spend time ramping up on areas of the product that they haven’t worked on before. When ramp-up time is not well tracked, the delays and impacts on productivity are often invisible. We’ve seen teams have to spend two additional weeks of “spikes” for learning, on top of working long hours and burning themselves out. This delay can cause the product or enhancement to miss a key opportunity in the market. And this hard-won knowledge may not be reused if the team keeps moving to new areas of the product. In addition, details about the product’s complex moving pieces are often lost in translation as one team hands off the work to another.
Reward the behavior you want to foster
Instead of optimizing your team for outputs, you should design internal incentives to reward work that improves outcomes for users. This can be a major change for an organization, in part because outcomes are tough to define and tougher to measure. How do we know if the quality of healthcare is improving? Even when outcomes are measured, it’s hard to tie our day-to-day choices and work to these outcomes. If users spend less time contacting support to resolve problems, how do we know if this is because the product is more self-explanatory or the result of some other factor like people relying more on vendors rather than struggling through the system themselves?
In order to address the challenge of shifting the measurement of success to outcomes, Ad Hoc works with government product owners to help define their OKRs (objectives and key results) for each initiative. We set up working sessions to provide structure and space for articulating North Stars — ideal future experiences for the users of the service. Once we establish a North Star, we work to refine the current year objectives — what are the most important goals to achieve in the next few months that take the service in the direction of that North Star, and what would qualify as success in reaching those objectives. These objectives then enable product owners and teams to prioritize the problems to solve, and provides a framework for measuring success based on whether the problem was solved.
While it may seem easier to frame objectives around task completion, it’s more important to define objectives around what the overall goals are. While it may be harder to measure some types of outcomes, it is possible, and in the end, will lead to better results. Key questions to ask when setting these objectives include:
- What experience do we want to create for our users?
- How will we need to change processes to develop that experience end to end?
- What groups and teams will need to collaborate to achieve these goals?
- Who is responsible for the end result?
- How will we know if we have achieved that result?
We have found that zooming out to see outcomes, reaching across silos, and owning problems that may otherwise fall through the cracks help us meet the ultimate goals of our customers.
To scale, someone must own the outcomes
Setting high-level, outcome-based objectives is one challenge. Aligning many teams to contribute to those objectives based on their specific capabilities is often an even tougher challenge.
For instance, one of the authors once encountered a situation where she needed to work across several different departments in order to enable a new user feature. It required work from developers, designers, IT, support desk, and learning and development. While the individual team members supported the goal behind the user need, they were constrained by departmental objectives. For instance, IT had committed to a limited set of out-of-the-box functionality to save money, but this functionality only served internal needs, not customer-facing ones. Because of the importance of the customer-facing need, people found workarounds, but the lack of shared objectives and a leader to champion those objectives added friction and slowed development.
What the organization needed was a clear decision maker to act as a sort of air traffic controller, aligning teams, setting priorities, and making sure everyone worked together for a shared goal. In government, each part of a product may have its own contract with its own product owner, but the product also needs an empowered product owner to oversee the entire effort and set the high-level outcome goals for every team working on the product. That person should also work to set internal incentives that encourage and reward teams for collaborating across the boundaries of their contract or project to further the overall outcome goals of the product.
When someone is clearly responsible for the overall outcomes, they’re incentivized to remind teams of the big picture and to steer them toward working together for a consistent experience. When this ownership is lacking, the results we’ve seen include front ends and APIs that don’t work well together, teams having to reinvent the wheel every time they develop a web form, and overall poor user experiences in tools that also don’t meet organizational needs. When we’re narrowly focused, we lose the forest for the trees.
Closing the gap
At Ad Hoc, we work with government agencies to help them close the gap between consumer expectations and government. That gap isn’t measured in the number of features or integrations. It’s measured in the experience of people using a service and the outcomes a product can enable people to achieve. To make serious progress on closing that gap, agencies and organizations need to set their internal incentives to focus on outcomes for users rather than outputs. While outputs can be an important accountability tool, when they take the lead they can cause perverse incentives that harm outcomes for users.
In our experience, agencies that take the time to set outcome-based North Stars, align their internal incentives to reward outcomes, and empower their product owners are better prepared to build excellent products for the public.
