Using performance testing to build resilient, accessible systems for all

A well-maintained, 10-year-old car will work fine to get you from A to B. A well-maintained bike will probably last even longer than the car. A 45-year-old landline phone can still receive and place calls. But what happens if you try to access a web page using a 3-year-old smartphone? How about a 5 or even a 10-year-old one?

It’s a given that the tech world moves fast – innovation is often synonymous with speed. The innovations that we’ve seen in web technologies usually come alongside increases in the memory and processing power of our devices. As the iPhone gets more powerful, websites use more resources.

But not everyone has equal access to computing power or connectivity. When websites are developed and tested against the latest and greatest, people with varying access to computing power and connectivity can be forgotten, excluded from some online space, or unable to complete a necessary task. Because of this, performance testing must be considered a key aspect of accessibility. It’s a powerful tool for enabling resiliency and has its place among other accessibility tests.

Performance and resiliency as part of inclusive design

A facet of Accessibility Beyond Compliance is ensuring that no one who needs access to some service is excluded for want of computing power. If a website is WCAG 2.2 compliant but folks need a gigabit connection for every page load or megabytes of spare RAM to render a page, it isn’t accessible.

This is especially important because computing power, like so much else, is becoming more and more unequal – between rising costs of the devices themselves, the power needed to manufacture, charge, and run these devices, chip shortages, and more – we can’t assume that everyone always has access to the newest technology or constant connection to the web.

Accessibility Beyond Compliance can help build resilient systems by ensuring that more folks have access to the services they need with the technology and connectivity they have at hand.

A resilient system is one that can recover and adapt to change or unexpected circumstances. Resiliency is an important consideration when working toward Accessibility Beyond Compliance because it empowers us to develop and design systems that can deal with circumstances we can’t predict. Here, we outline a few pragmatic steps you can take to start considering performance as a key facet of your accessibility tests.

How to get started with performance testing

Know your audience

Understand the folks using your software or service. This includes both the ones using your service today and those who will be using it in the future. Knowing your future audience is important because we can’t assume a website or service will receive updates in perpetuity. Funding may slow down, or priorities might shift. Even if your service isn’t receiving updates, it may still be a pivotal piece of infrastructure that people will rely on well into the future. So, know your existing audience, but also imagine the audience of the future to ensure access continues for them.

Set a performance budget

A key to successful performance testing is setting boundaries; the first step to performance testing is to set yourself a performance budget. This defines the minimum acceptable speed and responsiveness of a system given a known state. A well-defined performance budget allows you to prevent regressions and ballooning page load times as you develop new features.

A performance budget does this by defining a few important boundaries:

1. A minimum threshold for supported hardware
2. The maximum time it takes your software to load and run, also known as “response time”
3. Target size of data delivered to folks using your software or service
4. Assumptions about minimum supported network configuration

Research and analytics can help inform what your performance budget will look like, so don’t set arbitrary limits on bandwidth and supported hardware that may lock out people who rely on your systems.

Consider hardware

For teams getting started with performance testing, we recommend setting an achievable minimum threshold for supported hardware and then locking it into that threshold for a certain period of time – don’t cast old devices off just because a new version of a phone was released. Your tail of older devices should grow longer than the head of newer devices.

Consider connection

Network considerations are just as important as hardware considerations. Setting clearly defined performance thresholds given known network configurations allows you and your team to test your service using simulated network connections that might not match your own. This helps to ensure that your system performs well even with slower internet speeds.

Know your tools and understand how your performance budget is spent

Once you know your audience and you’ve set your performance budget, get to know your tools. Many utilities can make performance testing easier. One of the best places to start is to familiarize yourself with Runtime Call Stats; these can help you gauge performance. Another tool, Device Mode, helps you simulate mobile devices with different types of connectivity.

It isn’t all about finding a technical solution, though – some assumptions about your system and how it is used can lead to bloated solutions that result in poor or unequal performance. The key to understanding performance is ensuring that you and your team understand the system itself. This doesn’t mean that every team member needs to understand every intricacy of the system, but everyone should understand its complexities and the assumptions a system is built on.

An example of how to start to grow an understanding of complexities and assumptions is to consider what an average user session looks like. For a given user:

• How many actions are performed?
• What is the average load time for each action?
• What is the volume of data transferred to and from the device with each action?

Answering these questions allows you to start to see how your performance budget is spent. And with this understanding, you’ll be able to start to map what assumptions and complexities are impacting your system. Note that assumptions may be external to your system – you may have assumptions informing your choices about who is using your system, for example.

Building for reliable access – now and in the future

By including the steps above as part of your performance testing, you’re empowered to deliver systems that are resilient to a variety of circumstances – some that we can easily predict today and some that we can’t. Performance is a key aspect of accessibility because if a page fails to load, no one can access it. By including performance as a criterion for success when considering accessibility, you can help to ensure your system delivers the necessary functionality to anyone who relies on it.