The 21st Century IDEA Act Playbook Part 4: Prioritize modernization with user research

Earlier posts in this series have addressed the importance of designing around user needs. As researchers, we focus on understanding those user needs and translating them to the broader question of “how do we use this knowledge to build digital services that are both truly usable and engaging for people?” Our view of “government for the people” means ensuring that everyone who needs to access a service is able to use it. We’ll go through a couple of techniques below that we’ve used with Ad Hoc customers to identify priority services and ensure they’re usable and engaging for the public.

Include all your users

The first step agencies should take is to define the users for each of their digital services. This means taking a broad view of who users and potential users are. Unlike the private sector, which prioritizes value creation by focusing in on specific user segments, government websites need to provide for everyone. This means agencies must consider rare “edge cases” as well as the needs of users who may have permanent or temporary disabilities. That doesn’t mean that you shouldn’t prioritize the needs of the majority of users, but it does mean that all potential users are accounted for.

For example, when working on VA.gov, one of our team members visited several Veterans with vision impairments who rely on screen readers to navigate websites. They learned that while the global navigation for the site technically met Section 508 accessibility rules, people who used keyboards to navigate had to move through multiple levels of headers to access what they wanted. Including these Veterans in our model for VA.gov users and testing the product with them ensured that the product was not only compliant, but easy to use for real people trying to accomplish goals.

Understand how people use your services

You can begin to understand which sites (or which portions of sites) are most used with analytics. Most existing government sites have some form of digital analytics in place, and the Digital Analytics Program both mandates data collection and provides resources to make it free and easy for government agencies to put the tools in place. Analytics tell us which parts of websites are most used, where users are coming from, and where users are dropping off (which can be an indication of where they are challenged by a website).

Metrics and analytics can tell you a great deal about usage, but you need qualitative data to both understand the whys behind that usage and to understand the users and their needs and intentions. In other words, who are the people the digital service is serving, and how can that service best meet their needs?

Qualitative, direct interactions with end users can enable your agency to make sure the sites you build are usable and useful. They should enable people to achieve their goals, whether that’s enrolling in a health plan, finding Veterans’ services, understanding tax issues, or any of the myriad services people rely on the government for. We often use semi-structured interviews to dive into participants’ own perspectives and experiences with websites, tools, and services. Open-ended participatory methods allow users to interact with various artifacts to stimulate conversation and feedback. For instance, we conducted 1:1 card sorting exercises to understand what information is most important to beneficiaries when deciding on Medicare plans. Once we have a prototype, we will often do usability testing to learn what parts of it are working, what parts present challenges, and, crucially, why these aspects are problematic for users.

Gathering all of this information, both quantitative data from analytics and qualitative information from in-person user research, gives you a much better picture of how people interact with your services in the real world. Using this data about what works and what doesn’t, what’s highly trafficked and what’s rarely visited, is a much better way to prioritize your modernization strategy than agency strategy documents that may be out of date and disconnected from on-the-ground user needs. You should certainly include internal stakeholders in this process, but let users (both internal and external) guide how your agency modernizes its services.

Other considerations

Gather user information wherever you can find it. On the Hospital Quality Reporting program, which is redesigning the suite of applications hospitals use to publicly report quality measures and scores, the research team was sent call center logs. Our colleague carefully went through to analyze the issues users were experiencing and how they reported them. This allowed us to understand the most common problems, see how frequently they occurred, and create a visual representation of the issues for stakeholders.

Don’t overlook internal or specialized users. While the IDEA Act is focused on “public” engagement, many of the government digital services that need to be modernized are for specialized use and not accessible to everyone. The smaller user base makes them no less important to ultimately serving the needs of the people. For instance, on the Quality Payment Program, our work helped eliminate at least 40 million hours of burden through 2021, giving the time back to providers and suppliers to spend with their patients and not on paperwork.

Don’t let the Paperwork Reduction Act stand in your way. Many people familiar with the Paperwork Reduction Act worry that its provisions set limits on the kind of research we’re talking about, since it requires that all federal websites request permission from the Office of Management and Budget before collecting information from 10 or more members of the public. However, there are many exclusions to the regulatory definition of “information” under the Act, and the methods we’ve discussed qualify for these exclusions. Find more information in the new guide to the Paperwork Reduction Act.

Prioritize for your users

The Digital Services Playbook says that “The needs of people — not constraints of government structures or silos — should inform technical and design decisions.” As agencies prioritize their digital services for modernization in accordance with the IDEA Act, they should ensure it’s done with users at the center. In our research practice, we’ve found that researching a broad range of users with a mix of quantitative and qualitative methods helps teams better spend their resources and build better products. Prioritizing based on user data enables you to meet the IDEA Act mandate and to modernize in ways that will truly serve the people your agency serves.

Read part 1, part 2, part 3, and part 5 of the 21st Century IDEA Act Playbook.