The Ad Hoc Accessibility Beyond Compliance
Playbook

Designing with an accessibility-first mindset means focusing on people with disabilities as early and often as possible in the design process through rounds of iteration. It requires a design team to be intentional about addressing and prioritizing the needs and preferences of disabled people and other historically marginalized and underserved groups.

Play 1 Checklist

Ideation:

• Set up ideation and/or design thinking workshops to use accessible methods and tools with the intention of including and centering people with disabilities.
• Frame ideation around functionality and accessibility before delight and aesthetics.
• Review all ideas that will be translated into prototypes with an accessibility specialist.
• Include diverse perspectives and experiences in user stories, models, and artifacts.

Prototyping:

• Include proactive solutions for issues and concerns raised in research with disabled people (play 2).
• Design for mobile and assistive technologies first before designing for non-assistive needs.
• Design for multiple input modes including speech, keyboard, mouse, touch screen, and more.
• Annotate design artifacts like wireframes and mockups to help developers create accessible solutions. For example, noting heading levels, writing alt text, and identifying ARIA patterns.
• Write content and use images that are inclusive and/or representative of a diverse range of abilities.
• Ensure language is easy to understand at a grade 7 level or below and contains only what someone may need to complete their task.

Testing:

• Complete an accessibility review with an accessibility specialist ahead of conducting usability testing with real users.
• Design prototypes to be usable by assistive technologies when conducting usability tests. For example, using CodePen to create an HTML prototype.
• Test products with people with disabilities across a range of assistive technologies, devices, and experiences as early and as often as possible.

Play 1 Key questions

• How might we design for accessibility before delight?
• Have we considered users who may not have a mobile device, high-speed internet connection, or reliable access to digital services?
• How might we intentionally frame ideation, prototyping, and testing around historically marginalized and underserved communities?

Play 1 Common barriers

• We can’t test with disabled people because our team is structured around testing in Figma, Sketch, or other inaccessible prototyping tools.
  

  Counterplay: Consider using rapid CodePen prototypes or contextual “think aloud” studies (Nielsen Norman Group).

• As a remote organization, we can’t test with people with slow internet connections.
  

  Counterplay: Consider if you need to have a video of the participant while testing. A phone call or messaging app may be more accessible to some participants. Alternatively, consider collecting feedback asynchronously through a diary study or survey.

Conducting research with disabled people upfront can inclusively reframe how your teams define and approach problem spaces by:

• Giving a voice to those who may benefit the most from accessibility in the solutions you deliver
• Enabling your teams to learn more about accessibility and inclusion from interacting with disabled people
• Generating more inclusive solutions that appeal to a wider range of audiences
• Mitigating costly risks of having to fix foundationally inaccessible solutions

Play 2 Checklist

• Review research plans with an accessibility specialist to strategically recruit people with disabilities across a range of assistive technologies, devices, and experiences.
• Partner with organizations and government agencies that serve disabled people.
• Use tools to recruit participants that meet WCAG 2.1 AA compliance standards.
• Select flexible research methods that can accommodate a wide range of physical, cognitive, time, and economic needs.
• Pay participants for their time and expertise.
• Contact participants ahead of their session to understand their technical needs, check if they need accommodations, and confirm how they would like the session to be conducted.
• Provide foundational disability etiquette training for researchers and observers.
• Conduct pilot tests with an accessibility specialist.

Play 2 Key questions

• Who conducts research on your team? How can you check their biases during data collection and analysis?
• Who might be excluded or negatively impacted by your product or service? How might you take steps to mitigate this?

Play 2 Common barriers

• We don’t have control over who we can recruit.
  

  Counterplay: Document and communicate what types of disabilities and assistive technologies are missing from your research along with any associated risks.

• There’s not enough time or resources to conduct a study with disabled people.
  

  Counterplay: Document why there was not enough time or resources to conduct the study and address it in the next research effort. Contact accessibility specialists for an accessibility audit, heuristic evaluation, or general guidance.

• We don’t have an internal accessibility specialist to help review plans or conduct pilot tests.
  

  Counterplay: Reach out to your accessibility office or try contacting local disability non-profit organizations. You can also reach out to us for a professional consultation at hello@adhocteam.us.

• We can’t pay our research participants
  

  Counterplay: Explore ways to reframe an investment in research participants as a way to increase a user base or conversion metric.

​​Design systems are a powerful tool for product teams to ensure they’re building consistent and inclusive experiences.

By establishing a design system — a set of reusable components — that have accessibility baked in from the start, you scale the impact of accessible components. This alleviates the need to reinvent the wheel by providing designers and developers with building blocks and effective practices ready to be consumed. Great examples include:

• U.S. Web Design System (USWDS)
• GOV.UK Design System

Using a well-built design system multiplies the time and money saved for teams across a product because it helps to safeguard the consistency and quality of your components.

Note that a design system is a process and a living organism that will need to be updated and can’t be thought of as a one-off project.

Play 3 Checklist

• Determine a process where anyone can contribute to the design system in a structured way. Also determine who will handle governance, such as oversight, change requests, and new components to be added.
• Design reusable UI chunks (components) while considering constraints and how these elements will work with/around other UI.
• Build, test, and deliver accessible front-end code that is flexible across browsers and usable by all assistive technologies.
• Create guidance on how to implement components in an accessible way.
• Publish design files containing the final components (via Sketch, Invision, Figma, etc.) for download and/or syncing.
• Support people in learning to use it and help new teams adopt it.
• Prioritize including accessibility guidelines into each component’s documentation to ensure the components are being built properly and to promote accessible design thinking among teams that use your design system.
• Ongoing maintenance of the design system to ensure your standards are being met.

Play 3 Key questions

• Do we already have a suite of components or are we starting from scratch?
• What components can we tackle first that will have the greatest impact for end (product) users?
• Have existing components been tested to see how accessible they are?

Play 3 Common barriers

• There isn’t a dedicated team to work on this.
  

  Counterplay: Consider volunteering to be the point person on this and work to find like-minded people who can help you — becoming that dedicated team whether as an actual separate team or simply the ones that do that work.

  Counterplay: Talk to your leadership about the benefits of a design system (time and effort saved) to help get buy-in to begin a design system team.

• Will this really save time and money in the long run?
  

  Counterplay: While it’s true that the initial setup time for a design system will require time and effort, the payoff for this investment, in the long run, is guaranteed. Even something as simple as inconsistent buttons can create hidden, costly expenses and the headaches can multiply with more teams and more complex UI.

Testing your front-end code before it gets to production is a hallmark of a healthy accessibility strategy. Testing can be done in various forms, but here we’re addressing continuous integration and continuous deployment testing. This kind of test can help identify and capture 35-40% of basic accessibility issues and allow developers to be able to identify and fix code before it is merged.

Automated testing should run often; ideally, code is evaluated once a day. This encourages good coding practices and helps ensure a baseline of quality.

Tests should be atomic in nature, which helps to identify what went wrong. Since an atomic test focuses on a single feature of UI, the outcome is that you have a clear idea of what needs to be fixed.

Tests should include a concise error message when something fails or goes wrong. Developers should have ready access to the error message and be able to recreate the error on a local instance. This will confirm the error is not environment-specific and make fixing the issue simpler.

Whenever possible, tests should run on a server. It creates a more impartial testing environment. Tests can’t be skipped or modified easily, plus you get a trail of who might be bypassing the checks.

Play 4 Checklist

• Developers write tests for their code.
• Tests are run without human interaction beyond proposing a change to production code.
• Inline tests are run for new code before it hits production.
• People requesting a code change are notified when tests pass or fail.
• When one or more tests fail, pull requests are revised until all errors are fixed and all tests pass.
• Tests evaluate WCAG 2.1 success criteria.
• Tests are run on individual code components, user interfaces, and full page views. Unit tests are used for smaller blocks of code such as JavaScript functions, and integration/end-to-end tests are used for larger, composed blocks of code.

Play 4 Key questions

• Is this a new project or is this a project with an established code base?
• If this is not a new project, is there a build pipeline such as continuous integration, and possibly continuous deployment, in place?
• Does the continuous integration/continuous deployment (often referred to as CI/CD) pipeline include automated accessibility tests?
• What is the level of code coverage for any accessibility tests? Ideally, all pages and individual components need to be run through accessibility tests. At the component level, this is where using a design system that is thoroughly tested for accessibility is so powerful.

Play 4 Common barriers

• Incomplete or non-existent accessibility tests
  

  Counterplay: Start small. Work with whatever resources you have to break this work into doable chunks.

• Old versions of libraries, legacy applications, or rulesets (can cause false positives or miss legitimate errors)
  

  Counterplay: Collaborate with your product team to plan how to incrementally work through these dependencies. This may take some time to accomplish, but reconfiguring this old code will prove beneficial.

• Constrained timelines to launch or deliver new features
  

  Counterplay: Consider an MVP (or V1, V2, etc) roadmap to tackle the code that most affects, and is beneficial to, your product.

Accessibility Beyond Compliance performs at its best when teams are (a) holistically aligned on the purpose of accessibility and (b) understand how to be proactive, rather than reactive, in their specific roles.

“Everyone on a team has a role and responsibility for integrating and maintaining accessibility, and accessibility needs to be taken seriously in every role.” — Deque University

Play 5 Checklist

• Align your team on the case for making accessibility a priority. They should understand that accessibility work is never done, and requires constant learning, accountability, and iteration.
• Schedule disability etiquette, inclusive design, and accessibility training sessions or lightning talks as part of your project’s kickoff and/or onboarding.
• A variety of communication channels (direct messages, office hours, workshops) exist for teams to consult with accessibility specialists.
• Opportunities exist for team members to observe live or recorded research sessions with people with disabilities.
• Team members understand the specific accessibility and usability standards they’re responsible for achieving.
• Each team has accessibility champions.

Leadership and Product Managers:

• Understand that accessibility needs today may be different from needs in six months and have plans to be flexible to those needs.
• Staff accessibility specialists and disabled people to keep up with growth.
• Respond to requests for hardware, software, and testing devices required for inclusive research and accessibility testing.
• Provide vocal and written support for the work of accessibility specialists and champions.
• Regularly communicate about accessibility with team members to determine areas for improvement, continue fostering an Accessibility Beyond Compliance mindset, and evaluate accessibility integrations.

Play 5 Key questions

• How do your team members define accessibility? What are their experiences with accessibility?
• What incentives and business cases appeal to your team the most?
• How might you better support your accessibility specialists in terms of authoritative influence and work-life balance?
• How might you recognize and mitigate ableism when it occurs?

Play 5 Common barriers

• We don’t have the time to educate our entire team on Accessibility Beyond Compliance.
  

  Counterplay: Focus on wide-reaching, high-impact methods that can inspire interest towards investing in Accessibility Beyond Compliance like a lightning talk at a kickoff event, empowering employees interested in learning more about accessibility to become accessibility champions on their respective teams, or hiring people with disabilities.

• We don’t have anyone who could provide accessibility training.
  

  Counterplay: Consult with an outside organization or take advantage of online educational resources like Deque University.

You can’t make progress on an initiative unless you know where you’re at now and where you want to go. The same holds true for accessibility. Establishing metrics can give your team feedback on whether your efforts produce desired outcomes. Start with simple, broad metrics before easing into more complex, detailed data.

To start defining metrics, think about what you want to impact:

• The speed at which you do something (time)
• The quality of your output
• How much money you spend on something

Think about whether it makes sense to track quantitative data or qualitative data for the metric you’re considering, and how that relates to what you want to change. Remember, when it comes to accessibility, there’s no such thing as perfect. Accessibility happens on a continuum; don’t let positive metrics lull you into concluding that your products work when they don’t. The best picture analysis comes from a variety of sources.

Some input and output metrics include:

• Number of leaders engaged in accessibility
• Number of teams that have accessibility objectives and key results
• Budget for accessibility training
• Number of disabled people engaged in research
• User engagement or total number of users (the more accessible it is, the more folks should be able to access it)
• User satisfaction (surveys, qualitative research)
• Usability (system usability scale)

Also, think about what team or teams will need to be responsible for tracking this metric and reporting on it. From there, you can determine what it might mean if you don’t meet your goals. How do you enforce metrics you need to meet and reward teams for meeting them?

Further reading and references:
Measuring Accessibility Outcomes
Accessibility Program Fundamentals: Choosing the Right Accessibility Metrics

Play 6 Checklist

• Structure requests for Proposals and Statements of Work to ensure accountability for the products and services associated with the contract to pass WCAG 2.1 AA (and AAA when relevant) success criteria at a minimum (for government).
• Stakeholders and the team define and adopt an accessibility defect rubric that guides how accessibility defects get handled.
• Stakeholders and the team define and adopt a set of accessibility metrics that outline "What accessibility means" for the products and organization.
• Add accessibility as part of the acceptance criteria for launching new features and updates.

Play 6 Key questions

• Where is the organization now in its accessibility journey? Where does it want to go? How does it define success?
• What metrics do you currently use to assess the performance of your product or service, the impact of changes in the process or practices used to build or improve that product or service, or the progress of your inclusion initiatives?
• How might you apply an inclusive lens to your current metrics?
• What can your organization measure to help determine how it’s doing in regards to accessibility?

Play 6 Common barriers

• We don’t have much data to guide our accessibility efforts.
  

  Counterplay: Start thinking through what data you might have. Implementing accessibility programs can mean that you can begin tracking data on your own. For example, if you establish basic accessibility training for everyone, you can track the percentage of people who go through that training.

Many U.S. federal government contracts require vendors to measure and report on the quality of their work, defined in a Quality Assurance Surveillance Plan (QASP). A QASP often contains compliance activities the customer organization believes contribute to the overall quality of a product or service.

While helpful, we see accessibility measures and expectations often extending beyond a contract’s quality assurance plan. We believe that agencies would be better served if they took measures to weave accessibility and inclusivity into every aspect of digital service delivery, supported by measurable, defined outcomes that are tracked throughout the life of a contract.

In addition to building a service that truly meets the needs of all users, agencies benefit through receiving deliverables from higher-quality vendors. Creating experiences that aim for accessibility beyond compliance requires mature product thinking, DevOps, and agile capabilities.

Play 7 Checklist

• Include objectives for diverse user research, automated accessibility testing, and plain language content as part of the statement of objectives.
• Set stronger accessibility expectations (e.g. WCAG 2.1 or WCAG 2.2 instead of WCAG 2.0) for work products and production software. Use these expectations as part of a definition of done for agile user stories.
• Define outcome-based measures as part of a contract’s quality assurance metrics.
• Evaluate the process and work product of a voluntary product accessibility template (VPAT).
• Evaluate past project code and processes for accessible outcomes and inclusive methods.
• Evaluate proposals based on their approach to including accessibility and inclusivity in the organization’s software development practice(s).
• Evaluate whether RFP response materials are accessible.
• Ask vendors for evidence of investments in building diverse teams, such as diversity sourcing plans, in RFP responses.

Play 7 Key questions

• How will people interact with the products or services defined in the contract? What are the inclusion and accessibility considerations important for the success of the project?
• What service outcomes do we want this project to achieve? How can we measure accessibility’s impact on those outcomes?

Play 7 Common barriers

• De-prioritization of accessibility practices
  

  Counterplay: Tie accessibility to service outcomes.

• Lack of expertise or personnel on teams
  

  Counterplay: Include accessibility expertise as part of job description(s) or include an accessibility expert as a dedicated key personnel role.

  Counterplay: Require code submissions or demo of accessibility review practices as part of a submission.

As an organization takes an Accessibility Beyond Compliance approach, the emphasis must move from building accessible products to building inclusive and accessible workplaces that can further inform, enrich, and scale those efforts.

Can you remember a time where you felt left out? Did you resolve to change or improve that experience? Did you get push back or get exhausted by the response? People with disabilities often face resistance to change or are told to pick their battles when advocating for inclusive work cultures and spaces.

Organizations can proactively prevent those feelings by architecting work environments designed for a wide range of abilities. If it’s everyone’s responsibility to be intentional about inclusion, no one has to feel left out. Some ways organizations can begin becoming intentional about inclusion include:

• Holding everyone, including leadership, accountable for fostering inclusive culture and spaces with the intent of designing more equitable work experiences for team members with disabilities.
• Encourage allyship without overpowering the conversation, which enables disabled people to feel safe to ask for improvements.
• Addressing and taking action on experiences that limit access. These might include anything from a lack of physical access (e.g., missing captions on a Zoom call) to education on topics like ableist culture.

Play 8 Checklist

• Create an environment where leaders in your organization are driving inclusion and accessibility.
• Create a roadmap towards proactively creating inclusive culture and spaces within the organization.
• Make accessible tools and materials the default expectation within the organization including, but not limited to: timesheets, onboarding materials, collaboration tools, communication apps, and more.
• Encourage inclusive communication across the organization with simple techniques, like adding alt text to images, sharing links that include target and purpose in their text, using captions during video calls, and more.
• Use inclusive communication tools and methods. For example, some teams use a “raise hand” feature within Zoom or Google Meet to reduce the cognitive overload of remembering who went last during stand-ups.
• Include accessibility, inclusion, and disability etiquette training in onboarding materials.
• Create spaces for people interested in becoming allies to ask questions.
• Maintain and empower protected, anonymous methods for people with disabilities to provide critical feedback.
• Foster safe spaces for disabled people in the organization like private channels, affinity groups, or employee resource groups.

Play 8 Key questions

• What can we do to improve the accessibility of our tools, processes, environment, and culture?
• What actionable steps can we take to embed accessibility and inclusivity into our team culture and activities?
• How do we acknowledge and support those who are thinking about and advocating for inclusion within the organization?
• How might we be proactive instead of reactive?

Play 8 Common barriers

• Leadership and colleagues lack awareness of the need for accessibility and inclusivity.
  

  Counterplay: Push for foundational awareness training through lightning talks and onboarding materials.

  Counterplay: Focus on the small things that make underrepresented people feel included. Ask them what those small things are and implement them.

• No accessibility employee resource group or safe space exists.
  

  Counterplay: Create an additional space outside of your daily communication tool, like a bi-weekly call or meet up, for people with disabilities and allies.

People with disabilities have the invaluable ability to speak from a first-person perspective about their experiences using inaccessible products and services. These lived experiences make disabled people more qualified to advocate for accessible outcomes, evangelize for Accessibility Beyond Compliance, and create allies amongst colleagues. The disability employment guidelines from the Office of Personnel Management are a good example that we can all learn from.

Hiring people with disabilities enables organizations to:

• Generate more innovative, equitable, and accessible services.
• Access a relatively untapped talent pool with 1 in 5 adults in the United States having a disability.
• Encourage teams to proactively break down ableist and exclusive tendencies in their culture leading to a more diverse and retainable workforce.
• Provide closer connections to organizations and communities of disabled people for more access to an even wider variety of perspectives.
• Mitigate public relations mishaps that often occur when people without disabilities design for those who do.

Play 9 Checklist

• Hire from organizations, universities, and job boards that are centered around accessibility like American Foundation for the Blind, National Federation of the Blind, National Technical Institute for the Deaf, A11ySlack, Inclusively, A11yjobs, Diversability, and more.
• Create and empower an inclusive hiring leadership position in your recruitment team.
• Ensure all websites and digital tools used for the job application experience are inclusive and exceed WCAG 2.1 AA success criteria.
• Where possible, design interviews to be inclusive to a full range of physical, cognitive, and economic abilities (for example, offering preferences for interview setup, breaks, and flexibility for homework assignments).
• Where possible, proactively offer accommodations to all candidates even if they do not ask for it.
• Audit internal materials like onboarding documents, benefit sheets, training programs, and timesheets with an accessibility specialist.

Play 9 Key questions

• Are there any physical or cultural barriers that may be preventing potential candidates from applying?
• Where are you currently posting or advertising your job positions?
• Have your hiring process and internal materials been audited by an accessibility specialist?
• How might the evaluation process of candidates be designed to mitigate ableism and other unintentional biases?
• Are you hiring from a diverse spectrum of disabilities? Hiring only candidates with low vision.

Play 9 Common barriers

• Absence of or lack of access to an accessibility specialist on a hiring or HR team (in private organizations)
  

  Counterplay: Partner with a third-party organization like Inclusively that specializes in hiring people with disabilities.

• Using 3rd party recruitment tools that are not accessible (in private organizations)
  

  Counterplay: Offer alternative ways to:

  • Apply by accessible online forms (Jotform, Google Forms)
  • Apply by phone. For those who may have difficulty with verbal communication, allow them to indicate their preferences in the application like offering a phone number and options like “Relay number, Text only, or other (with form field explaining their needs).” Offer training to the recruitment team on these options.
  • Apply by email
• Internal tools and materials like timesheets and onboarding documentation are inaccessible to assistive technologies.
  

  Counterplay: As a temporary solution, offer an optional human reader to assist in reading and using inaccessible internal tools and materials until permanent solutions are available. There should be enforceable guarantees that this will not negatively reflect on the applicant’s ability to complete the job.

  Counterplay: Contact teams responsible for provisioning internal tools and materials and create a plan to retire and replace inaccessible technology.