Teamwork makes the dream work: Managing effective product analytics projects

Working with data is complicated. From collection to processing, modeling, and visualization, there’s a long life cycle that data must go through before getting into the hands of the person who will learn from it. Further complicating matters is that technical expertise and problem solving are required at each step. However, one of the biggest challenges to making data actionable comes at the very end of the life cycle: the handoff between generating insights and putting them into action. The reality is that converting data into insights isn’t just a technical problem – it’s a process problem.

This challenge is common to any type of data that organizations want to use to influence behavior change. In software development, it happens most frequently with metrics that capture information about how users behave with a product or service – a process known as product analytics (think user traffic, button clicks, and feature engagement). Product analytics requires collaboration between two disciplines: product and data. When these roles haven’t developed a common language and don’t understand each other’s workflows, such collaborations can fail. But when data analytics and product management work together well, this information can help organizations prioritize work, identify pain points, and learn how users respond to new features.

To effectively manage product analytics projects, we must be aware of common issues that can impede progress and apply best practices to set teams up for success. We also need to understand the key roles that play a part.

Who does what?

Data analysts are responsible for getting the right data in front of the right people in ways that are easy to understand and act on. They know what data is available, how it is collected, and its potential limitations and blind spots. Analysts have the technical skills to properly model data, analyze it, and build engaging visualizations for data consumers. They are also vital collaborators in key performance indicator (KPI) selection. They use their knowledge of the data to evaluate if a potential metric is robust enough not to be impacted by ambient noise in the data, while still being sensitive enough to show an effect when important changes are made.

Product managers (PMs) are responsible for ensuring that the highest-value work gets delivered. Because PMs work across functions and stakeholder groups, they develop a sense for the political and operational constraints around getting work implemented. The on-the-ground nature of their work leads them to learn about the historical and user context behind their product portfolio. Organizations vary in terms of how data savvy product managers need to be. In some rarer cases, querying data for analysis might be a regular part of a PM’s responsibilities. However, it’s typically more common for PMs to have the analytical skills to interpret data, while other people own earlier parts of the data life cycle.

Because PMs play a key role in product decision making and stakeholder advocacy, they can play a “champion” role for data within an organization. When PMs are empowered to use data, they can help disseminate insights and make informed bets. However, without effective product/data collaboration, PMs may over-index on intuition or user research to make decisions, which can result in missed opportunities or misguided bets.

Symptoms of collaborations gone wrong

Even when data and product teams work together, not all collaborations are set up for success. While each project may face specific challenges, there are common symptoms we might see that indicate a product analytics partnership has gone sideways and is in need of realignment.

Insights from analysis not implemented

In this scenario, a data tool provides potentially actionable insights, but there’s a disconnect between these insights and what the product team actually decides to do. This is usually caused by a disconnect between what the data team built and what the product team actually needs.

For example, when an analysis, dashboard, or other data tool doesn’t line up with the product team’s immediate goals and roadmap, they may not see the value of applying its findings to their work. Similarly, when analysts surface insights that don’t take the product team’s constraints into account – whether due to costs, timing, political influence, or resources – these are understandably less likely to be put into action.

The best remedy for this situation is to create time for data and product teams to get on the same page before the next iteration of their data tool. By front-loading collaborations with real-time discussion and product resources, analysts will have a clearer understanding of the team’s immediate goals, the types of decisions they need to make, and the scope of the work they can control. This ensures that data products visualize the right metrics in a way that supports discussion and action.

Low engagement with data tools

Another sign that product analytics isn’t going as planned is when dashboards or other data tools have been requested and created but aren’t being used. Logs show that members of the product team aren’t viewing the tool, and demos or other presentations make no mention of it in the team’s decision-making process. This can lead to frustration on both sides of the table – data analysts see their work go unused, and product managers see their toolkit functionally stay the same.

Similar to the scenario above, this may be due to insufficient active collaboration time between data and product teams at the start, resulting in the wrong thing getting built. And even if these initial meetings did happen, we might arrive at the same outcome if the people who will actually use the tool weren’t involved in the process of deciding its requirements.

There can be other drivers of low engagement – lack of trust that the data has been transformed correctly or not enough familiarity with the tool or platform to understand how to use it. When either of these is the case, making sure the product team has access to good, clear documentation can go a long way in lowering barriers to engagement.

General friction

Before a data tool has even been delivered, tensions may start to rise between product and analytics teams. In fact, if left unchecked, the collaboration may stall out and the data tool won’t be delivered at all.

The root cause of this is often delays, deadlines, or roadblocks that one team runs into that the other is not aware of and feels blindsided by. For example, the product team may need to have the data tool completed and in use before they can launch a new feature. If the data team experiences delays, this could be a major blocker that erodes their trust in the collaboration. On the other side, the data team may need to secure new data sources and complete complex transformations to give the product team what they need. This may take more time than anticipated.

To preempt these challenges, teams should work proactively to understand each other’s work so that delays are not surprises and deadlines are shared stakes.

Best practices for strong collaborations

Whether you’re starting a new product analytics collaboration or trying to get a current one back on track, good partnerships require a well-structured, traceable process. We recommend prioritizing the use of templates and workflows that support a human-centered design and agile approach:

• Work together early. Front-load your real-time discussions to build understanding, ask questions, and refine ideas for KPIs or experiments. Ahead of these sessions, PMs should do the legwork to set analysts up for success. For example, PMs should consider outcomes or questions they want to answer and brainstorm examples that will help analysts understand their goals. And analysts should ask many specific follow-ups to make sure their technical definitions of metrics match the product team’s.
• Work together often. Plan for multiple opportunities for iteration. Analysts should provide product teams with updates on KPI selection, technical definitions, and dashboard mockups to get feedback and course correct where needed.
• Create tools to gather complete info. To help product teams request new data pipelines or dashboards, data practitioners should create templates and request forms to improve collaboration with product teams. This helps clarify exactly what is needed and reduce turnaround times.
• Keep work out in the open. Require good documentation and observable quality assurance processes to build user trust. This way, discussions can be less centered on “are these numbers right?” and more on “what should we do with this information?”

In addition, collaborating teams should understand each other’s workflows, constraints, and goals. Product teams should have an understanding of analytics workflows. This includes knowing the basic steps that need to occur to transform data and build new tools – and where delays are most likely to happen.

Meanwhile, analytics teams should have an understanding of how their work fits into the product team’s roadmap: what are the decisions they need to make on what timelines? Laying this groundwork early will help reduce surprises and improve collaboration for all involved, especially if everything doesn’t go exactly as planned.

Whether you’re a product manager about to embark on a new product analytics project or a data analyst looking to shake up the way your team engages with others, these recommendations should help you set the table for strong and streamlined collaborations.