Setting the right benchmarks for site speed in government

At Ad Hoc, we believe the government has a duty to prioritize equity and accessibility in service delivery. In practice, that means the goals and benchmarks used in the private sector often won’t apply to public digital services.

While the bulk of guidance in the broader tech industry is focused on how large private web applications measure success — with a focus on revenue and outperforming competitors — we find that those measures often don’t align with the goals or intent of public government services. One example of this is in how we approach web performance — the speed and file sizes at which a site or app delivers content to users. This is measured according to the total file size a device must download per page, and the time it takes for that page to load.

In the private sector, the industry standard guidance is to set performance benchmarks by running a market analysis on your main competitors, then aiming to build a site that is 20% faster than the competition. When we talk about public sector digital properties, however, this approach falls flat. Most government sites don’t have competitors, nor should they be comparing themselves against them even if they do. So how should we be thinking about performance benchmarks if we aren’t measuring against the competition?

Most government sites don’t have competitors in the same sense as the private sector

If a Veteran wants to sign up for services from the Department of Veterans Affairs (VA), the only place they can go to do so is the VA itself. They may sign up by calling a contact center, going through VA.gov, or using VA’s mobile app, but these are not competing services. They’re simply different channels from the same organization.

Additionally, in an underfunded public sector environment, we regularly find that many similar service offerings within the same vertical or service domain are outdated. Again, we wouldn’t want to look at them as “competitors” to measure against or try to outperform. Outperforming outdated systems isn’t the right comparison.

When we’re talking about site speed and download sizes, the direct impacts are on the humans that use them, and the environment in which they’re used. That’s our motivation for improving site performance.

Poor web performance widens the digital divide

It’s quite likely that you’re reading this post from a computer connected to broadband internet. Or at least a mobile phone with a fast connection. In that environment, page speed is like oxygen — you don’t even need to think about it. So let’s back up and talk about where speed and page weight does make an impact. Spoiler: it impacts people on the margins; those who are most likely to need access to government services.

Not everyone has access to home broadband

In 2021, 15% of U.S. adults were “smartphone-only” internet users – that is, they have a smartphone, but do not have a home broadband connection. Those who are “smartphone-dependent” are often limited by slow data speeds and spotty availability. People who are smartphone dependent are also more likely to fall below the poverty line, which means they’re also more likely to need government assistance.

On the whole in 2021, 22% of Americans were still without access to high-speed internet. It’s within these populations where page size can have a big impact on site speed, and where a lack of consideration to page weight can have equity impacts.

Smartphone-only users are also more likely to run up against data caps on their mobile service plans. For someone watching their data limits, every unnecessary font or carousel image a site forces on them may have a real impact on whether or not they can complete the tasks they need.

Slow page loads can have a cognitive and health cost.

Usability experts have found that if a page takes even 1 second to load, visitors start to think about something else. Somewhere between 1 second and 10 seconds, those users will bail altogether. If those visitors give up on an online service because it’s loading too slowly, this may result in lower adoption rates of the service altogether.

Research has shown that waiting 10 seconds for a page to load can cause a 38% increase in heart rate. That’s comparable to watching a horror movie, and worse than the stress of waiting in line. Users with memory or cognition challenges may struggle even more to maintain focus on the task at hand if the service they’re trying to access doesn’t load quickly enough, again leading to the potential of abandoned user flows.

Setting speed benchmarks for equity

As we consider the importance of building products and services that meet the needs of users and provide equity for all, we want to focus on speed and file size benchmarks that consider those human costs of bloated pages, rather than how we compare to other sites.

We recommend setting a performance goal of less than 5 seconds load time over a 4G connection for public-facing apps. This translates to about 100KiB of HTML/CSS/fonts and 300-350KiB of JS (gzipped) as carefully detailed in The Mobile Performance Inequality Gap, 2021. This benchmark considers the realities of modern web development while also optimizing for the experience of the audiences for whom we’re developing. The factors that can go into reducing load time could be a series of blog posts on their own, and Digital.gov has a good primer. Some things you can do include compressing images, minifying Javascript and CSS, optimizing and caching backend systems, and removing unnecessary dependencies (like GOV.UK did recently when they removed jQuery from frontend apps). On the whole, tracking and improving site performance should be a part of a product team’s incremental updates, along with reducing technical debt and improving user experience.

As we think about ways in which government sites meet the needs of their users, speed is an integral part of what makes a site inclusive and accessible to more people. As consumer apps are simply measuring themselves against each other, they may end up setting less aggressive benchmarks than they would if they were truly looking to serve the widest audiences. By aligning our performance benchmarks to human experience, we can exceed consumer expectations and lead the way to better services.