Decommissioning Cohorts

Cohorts is Ad Hoc’s custom-built user research participant management application. Since its launch in March 2017, Cohorts has been used at some point by nearly every research team at Ad Hoc, with heavy use from our Veterans Affairs and Hospital Quality Reporting research teams. Our teams often work with vulnerable populations, so there are security and regulatory compliance concerns on how research participants’ data is stored and accessed. Because Cohorts was developed in-house, with limitations on who could access it, and its data owned by Ad Hoc rather than a third party, it was an attractive solution to our research teams’ needs and our government partners’ requirements for security and regulatory compliance.

However, we’ve run up against Cohorts’ limits. Through the life-cycle of our contracts, our research becomes more granular and our needs more specific. Senior Researcher Elissa Frankle Olinsky says, “As the research teams have grown and Ad Hoc’s research opportunities have expanded, we’ve found needs for participant management that extend beyond Cohorts’ capacity.” For example, our teams needed sophisticated search capabilities and form integration that Cohorts lacked. In sum, we have outgrown our current tool.

This is a fundamental trade-off in building and maintaining a tool in-house. It can take significant product and development effort to address its users’ requirements, like any software product. We could double-down and spend design and engineering resources continually improving Cohorts into the tool we need. Our core business, however, is in providing those resources as services to government. So instead we made the tough decision to focus our efforts on our customers, decommission Cohorts this month, and find a tool with the user management we need out on the market.

Our first step will be to migrate our participant data into an interim solution while we search for a new platform or platforms that meet government and project compliance on important features like data security, controlled access, health privacy laws, and contract specific concerns.

We plan to share updates as we learn more tooling options for user research and our experience with them. User research in digital services is a maturing field that has specific product needs, and we look forward to exploring this space and sharing what we learn.

Illustration by Barb Denney