Product Reviews: Building a culture of feedback loops in government

But time, energy, and space don’t always exist for the repetition and practice that are key to product managers’ growth. So, how do we help product managers strengthen their product skills and develop a shared understanding of high-quality product sense?

We created Product Reviews to provide product managers with non-judgemental feedback on their strategic approaches from cross-functional peers. In Product Reviews, product managers frame the product problem, explain their hypotheses, define measures of success, and field questions and suggestions. They come away with a more informed picture of the path forward and take lessons back to their teams.

Product Reviews are one of the important ways we invest in our product practice area. It ensures our product methodology has the opportunity to iterate and improve, just as an individual product does, so we can more effectively deliver better government outcomes.

What is a Product Review?

Product Reviews are one-hour sessions where one product manager presents a strategic product decision, and five to seven other members of Ad Hoc (product managers, engineers, designers, and researchers) offer live feedback through Q&A.

Presenters walk through the problem they aim to solve, the product outcomes they want to achieve, and their thought process on how to get there. Product Reviews can be applied to any product problem or process opportunity. For example, what does launch preparation for this new feature look like considering past go-to-market difficulty? Or, how do we increase the number of people within a certain population who apply for health care? Or, what’s the larger strategy for this portfolio of products?

After the presenter frames the problem and their proposed solution, reviewers prioritize feedback on presenters’ product thinking and not on how polished the presentation is. Given that reviewers come from different levels of seniority, different products, and sometimes different practice areas, we’re able to hear from many viewpoints within a single session. The presenter can look to a Product Review as an intimate, safe space for them to receive feedback on their strategy.

Why are Product Reviews helpful?

It’s easy for any product manager to tunnel-vision on delivery and the tactics and methods that they and their team have “always done.” We want to offer our product managers a safe space to get constructive feedback and tap into the collective knowledge of product management at Ad Hoc.

A unique aspect of the product work at Ad Hoc is the diversity of our product portfolio. At other tech companies, product managers may own separate products related to the same service. We benefit from work across different agencies and distinct products at various stages of their product maturity. Ad Hoc product managers aim to be leaders in product thinking and product strategy within government digital services regardless of the type of product we work on. Reviews offer a way to connect people across our practice area (and beyond!) to coach and mutually grow our product strategies to strengthen Ad Hoc’s product management as a whole.

What have we learned?

We experienced different iterations of Product Reviews. Originally, Reviews were framed as Product Critiques. We learned from participants that the name felt daunting and negative and went against our goal of a safe and judgment-free space. We emphasized that reviews shouldn’t require too much preparation in advance; the inputs to a Product Review are the types of artifacts and conversations that are already happening on individual teams. We provide additional space for that thinking.

We also learned that these reviews can be valuable no matter who you are: a seasoned product lead figuring out new ways to coach product managers, a designer or engineer looking to share their perspective, or a new product manager absorbing how Ad Hoc product managers think about product. All benefit from regular cross-functional feedback.

In the first few years, these Product Reviews were infrequent, which made it difficult for our product managers to rely on the reviews as a consistent way of getting feedback. Starting last year, we began offering monthly review sessions to make sure our practice area can count on a dependable venue for feedback.

Where do we go from here?

What started as a proof of concept a few years ago is now an integral part of our product practice area. We have a four-person committee focused on increasing opportunities for Ad Hoc product managers to review and get feedback on their thinking. We’re exploring different avenues of feedback to best accommodate the size and diversity of our product practice area in addition to hosting monthly reviews. Some product managers benefit from asynchronous feedback on specific deliverables, similar to a peer review process for product artifacts, rather than a one-time presentation. Others benefit from a simulated scenario that allows them to flex their product thinking in ways their current product may not support. And, we plan to solicit feedback from the product practice to ensure Product Reviews continues to help strengthen our collective product methodology.