Using AI to improve government data quality and decision-making

Artificial intelligence (AI) continues to create new opportunities for government agencies to solve practical challenges, improve efficiency, and make better use of the data they already have. When applied thoughtfully, AI can help teams automate repetitive tasks, identify patterns with large datasets, and uncover insights that would otherwise require significant manual effort.

For government, however, adopting AI isn’t simply about embracing new technology. It’s about applying the right tools to real mission challenges while maintaining the security, transparency, and human oversight that public services require. Used responsibly, AI can help agencies improve their operations, strengthen decision-making, and deliver better services to the public.

Using AI to improve data and decision-making

Government agencies manage massive volumes of unstructured information, including health claims, program data, reporting, and other qualitative records. While these datasets contain valuable insights, manually organizing and analyzing them is often time-consuming and labor-intensive.

In these kinds of scenarios, AI can be a powerful solution that can help agencies improve data quality and make information easier to use. Rather than replacing human expertise, AI can organize complex data and identify meaningful patterns so teams can spend less time managing information and more time acting on it.

A case study in improving data quality

Imagine trying to make program decisions using a database where one out of every three records is a duplicate. That was the challenge one of our customers faced.

The challenge

The agency used a centralized system to collect goals submitted by grant recipients to track their overall progress and inform future technical assistance. When we began supporting them, the system contained more than 50,000 goals, approximately 34% of which were duplicate or similarly worded.

Because the goals were entered as free text, duplicate records often differed only by capitalization, punctuation, verb tense, or spelling. Staff couldn’t easily determine whether similar goals represented the same need, whether previous assistance had already addressed an issue, or where future support should be focused. Manually reviewing and reconciling duplicate records across tens of thousands of entries was labor-intensive and made accurate reporting and program planning more difficult.

The solution

Traditional methods, like exact text matching, couldn’t reliably identify goals that expressed the same idea using different wording. Instead, we implemented a natural language processing (NLP) tool that analyzed the meaning of each goal and identified potential duplicates, even when the text varied.

The solution presented recommendations via an intuitive workflow, allowing users to review suggested matches, merge duplicate goals, or decline recommendations. AI accelerated the analysis, while people remained responsible for every final decision. By keeping users in control throughout the process, the agency improved data quality while maintaining confidence in the accuracy of its records.

The solution also provided proactive nudges as users entered new goals, helping prevent future duplication before goals were entered into the system.

The results

With AI assistance, agency staff merged almost 3,000 duplicate goals, reducing overall duplication by 51%.

But the impact extended well beyond cleaner data. Staff who previously spent hours manually reviewing duplicate records could instead focus on higher-value technical assistance work. Improved data quality gave the agency a stronger foundation for planning, reporting, and resource decisions. At the same time, the AI-assisted workflow continues to help prevent future duplication as new goals are entered.

Additionally, we’ve scaled this solution to fundamentally elevate agency oversight, compliance, and strategic decision-making. Using NLP-driven insights, we eliminated open-text ambiguity and built a standardized framework that unlocked seamless, cross-regional analytics for the first time.

To ensure long-term success, we put the solution to the test early. An 87% adoption rate during testing validated the framework while surfacing critical insights to fine-tune the user experience. By replacing heavy data burdens and manual oversight with a centralized goal dashboard, we empowered agency leadership to shift from reactive maintenance to proactive program growth.

Raising the bar of government services with AI

As expectations for digital services continue to evolve, government agencies have an opportunity to apply AI in practical, responsible ways that improve operations and strengthen public services.

The greatest value comes not from using AI for its own sake but from applying it to problems where traditional approaches fall short. By combining AI with human expertise and oversight, agencies can reduce manual effort, discover insights from their data, and make better-informed decisions while maintaining the trust and accountability that is essential to government.

At Ad Hoc, we’re helping government explore and implement targeted, responsible uses of AI that create measurable value for their programs and the people they serve.

If you’re interested in learning how AI can improve your agency’s data quality, drive more powerful decision-making, and significantly increase your agency’s efficiency, let’s talk.