Reviewing procurement responses for accessibility

Posted:
Tagged with:

We recommend where possible, you involve your organisations digital accessibility officers in the review of any supplier responses received during a procurement exercise. It is important to get the feedback of accessibility professionals to be able to correctly judge the quality of documentation provided by suppliers.

For example, if a supplier provides an accessibility roadmap it should be reviewed by an appropriate accessibility professional outside the project. They weigh up the potential user impact, proposed mitigations, and consider the availability of alternative products. They may approve or not; approval may be subject to specific mitigations being implemented as a prerequisite to purchase.

If you do not have accessibility specialists to review your supplier responses, this guide should help you understand the types of responses you might receive from suppliers and what you should be looking for in different response and documentation types.

Once you have read this guide, you may find more itemised support useful in scoring supplier responses. The MTA Procurement scoring and maturity guide and template can help you to begin turning the learning from this guide into practical steps during your procurement process.

This publication is issued as guidance only. It is not intended to provide legal or professional advice. All collaborators on this content accept no liability for any errors, omissions or any consequences, losses or damages arising from any use of, or reliance placed on, this publication. You should seek advice from a suitably qualified professional.

How to interpret accessibility responses

Assuming you have used the MTA procurement requirements template to get information from suppliers on how their systems will meet your requirements, you should have received some information which you need to assess to ensure that the supplier’s approach to accessibility fits with your organisational requirements.

Note: All suppliers are different and may record their accessibility information in different ways. The following guidance has been broadly grouped into common categories for ease of reference. The key thing to be looking for is level of detail in the supplier response and evidence of a robust approach to improving and maintaining accessibility requirements, whatever form that takes.

The below sections provide further information on what these responses might look link and how best to respond to or interpret different responses and evidence.

Our product fully complies

Many suppliers will claim that their product fully complies with all WCAG A and AA success criteria. In our experience this is most often incorrect, but not due to a lack of good intentions.

If you receive documentation which shows a clean slate for accessibility, be somewhat sceptical and ask to be able to do your own testing. As we often say, most of the time any suitably large system will not be perfect, and sometimes that must be accepted. You are looking for an accurate state of the system not clean documentation just to get past the check.

Most suppliers that have a fully compliant VPAT or other “clean” evidence have likely put in work to test their platforms. Ask the supplier about their testing processes and if/how they have included accessibility checking into the development cycle.

Suppliers may also answer follow up questions explaining that there are a limited number of areas where the product has current issues against WCAG. This might be in a written response to one of the tender questions or can be identified through documentation such as a VPAT or audit report as mentioned below.

If you receive a response from a supplier which says, “fully compliant” and then receive documentation which does not say the platform is 100% perfect, you should raise this inconsistency and double check the causes and what suppliers are doing about issues.

Consider:

If you find small discrepancies between their documentation and your own testing, this may be due to the changing nature of platforms, not a lack of effort. If the supplier has other supporting evidence, this may be a good response that can give you confidence in a supplier.

VPAT

Voluntary Product Accessibility Templates (VPATs) are a type of document which provides a very basic level of information on the accessibility of a digital product. This will normally be in the form of a table listing each of the WCAG success criteria, identification of their compliance, and some notes. VPAT's are part of the American Section 508 requirements but have become internationally used pieces of accessibility documentation.

The problem with VPATs is that they are normally not detailed enough to give useful information on the true level of impact for a service.

For example, a VPAT may say for a digital system that it partially complies with 2.1.1 Keyboard (A). The notes read “Some areas of the product are not accessible to all keyboard controls.” The question you should be asking is what does “some” mean? Is it the main navigation for the platform? Is it all main content? Or is it a logo in the footer? If the navigation is not accessible or the main content is not accessible but other content is, then that still counts as “some” but would mean that a wide range of users would not be able to interact with the product at all.

If you receive a VPAT that includes vague statements such as the above, which could mean a potential risk, or you are not sure, ask the supplier for more detailed information on the exact areas of the product that the issue covers. A VPAT can only have been produced correctly following in-depth testing. VPATs are summary documents and so the detailed test results must exist.

You should also be asking for a detailed remediation roadmap. If the supplier is already aware of several issues as detailed in their VPAT, ask them for evidence of how they are planning to fix those identified issues, or what actions they have already taken if the VPAT is more than a few months old.

If the supplier cannot give you more detail than the VPAT alone, be sceptical of the documentation. You know there are some issues at this point, but the supplier cannot give you any evidence as to the impact on their product or what they are doing about it. This can present a high risk and may suggest that getting accessibility fixes made may be difficult. Avoid the supplier in this case.

If the supplier can follow up the VPAT with more detailed documentation and a roadmap, or if the VPAT itself already contains more detailed specifics about the impacts of individual success criteria, and the supplier has a roadmap, then this may be a good response that can give you confidence in a supplier.

Detailed testing documentation

If the supplier responds with detailed testing documentation for the product, you should look through the documents carefully. But this is generally an initial good sign. You will want to look at:

If there are any issues present in the detailed documentation you should ask for a detailed remediation roadmap. If the supplier is already aware of issues, ask them for evidence of how they are planning to fix those identified issues.

If the supplier has no answer, cannot show actions of accessibility fixes, or does not want to commit to a roadmap, this can present a high risk and may suggest that getting accessibility fixes made may be difficult. Avoid the supplier in this case.

If the supplier has detailed documentation, and can show how they are testing, what they are testing, and what they are doing about results through a roadmap, this may come close to a model answer that can give you confidence in the supplier.

Accessibility Statement

The accessibility statement is the end of the documentation journey. If the supplier can provide you with an accessibility statement, this is an insight into their testing process. If the statement is good and has useful information on issues and even suggested workarounds already this could be a positive sign.

If the statement is poor and does not include specific information about technical issues and their effects on users, but instead includes vapid statements about "commitments to support all users" this could be a sign that the supplier does not have meaningful practical documentation to back this up and show what actions they are taking to deliver on those commitments.

In either case, if an accessibility statement is provided as the main piece of evidence, you should respond asking for other detailed testing documentation which must have been used to inform a good accessibility statement.

Remediation roadmaps

In the event that you receive detailed documentation from a supplier that includes identifying issues as well as remediation roadmaps which show when each issue is planned to be fixed, this is an excellent response and should show that the supplier is aware of issues with their product, are working to fix them, and can provide you as the customer with significant evidence to support due diligence.

At this stage, you can talk with the supplier about prioritisation and timelines. What issues has the supplier prioritised first? The biggest, most impactful issues, or the “quick wins”. Speak with them to better understand their current position and what mitigations and guidance material you may have to put in place in the interim period to support users while fixes are being made.

If any significant milestones within the roadmap have been passed, ask if items on the roadmap have now been completed, this is a good chance to see if the roadmap is a “tomorrow never comes” situation, or if actions are being taken and individual issues are being resolved.

Plans for testing

Sometimes, particularly with new products that are either being designed bespoke for you or are a new launch from the supplier, thorough testing and road mapping may not yet have taken place. In this case you would expect to see information from the supplier about how they plan to test the product and what they will do about issues found during that testing.

Ideally you will be looking for suppliers to commit to testing against the latest WCAG A & AA success criteria as a minimum. This can be done either in house if the supplier has the skills, or by a subcontractor such as professional accessibility auditing companies. In any case the testing should include:

If suppliers are providing you with plans for future testing, a good response must cover the points above. If the supplier suggests testing with only automated tools for example, this is not sufficient. As this is most likely occurring as part of a more bespoke offering you have the opportunity to demand that the above points for testing be mandatory as part of any delivery. At that point it is on the supplier to provide you with a compelling explanation that they understand and have the tools / skills to conduct that level of testing in-house or tell you who they may subcontract out to as an accessibility testing partner to deliver this.

No information or other negative response

Unfortunately, not all suppliers have model answers for accessibility documentation. Many may not even be aware of these requirements or may be new to this area of compliance. This means that for many responses you may receive sub-standard or no information.

If a tender response cannot provide any information to show the supplier is aware of the legal accessibility requirements for their customer base and know how to deal with requests and have appropriate evidence, then they are significantly higher risk.

The important thing is how the supplier responded to these requirements. If the supplier approaches it proactively and try to make fixes and are operating in good faith then that is one thing, but conversely, they may dismiss the need because it has been "unimportant" up till now. The supplier's direction from this point is important.

The range of poor responses you might receive from suppliers can be quite broad. You may receive answers such as the following, which are all real responses we have seen before. For each we have provided some notes on why these can be a problem for prospective customers:

Next steps, post procurement and contracts

Once you have read this guide, you may find more itemised support useful in scoring supplier responses. The MTA Procurement scoring and maturity guide and template can help you to begin turning the learning from this guide into practical steps during your procurement process.

Once you have completed your tendering process and have decided to enter into an agreement with a supplier, it is important to keep accessibility compliance in the requirements.

The best way to do this is to include accessibility into contracts with suppliers. For more information, read our guide on accessibility in supplier contracts which includes example contract clauses.

Share on:

TwitterLinkedIn

Site preferences

Please feel free to display our site, your way by finding the preferences that work best for you. We do not track any data or preferences at all, should you select any options in the groups below, we store a small non-identifiable token to your browser's Local Storage, this is required for your preferencesto persist across pages accordion be present on repeat visits. You can remove those tokens if you wish, by simply selecting Unset, from each preference group.

Theming

Theme
Code block theme

Code theme help

Code block themes can be changed independent of the site theme.

  • Default: (Unset) Code blocks will have the same theme as the site theme.
  • Light 1: will be default for users viewing the light theme, this maintains the minimum 7:1 (WCAG Level AAA) contrast ratio we have used throughout the site, it can be quite difficult to identify the differences in colour between various syntax types, due to the similarities in colour at that contrast ratio
  • Light 2: drops the contrast for syntax highlighting down to WCAG Level AA standards (greater than 4.5:1)
  • Dark: Syntax highlighting has a minimum contrast of 7:1 and due to the dark background differences in colour may appear much more perceivable

Motion

Motion & animation

Motion & animation help

  • Default (Unset): Obeys device settings, if present. If no preference is set, there are subtle animations on this site which will be shown. If you have opted for reduce motion, smooth scrolling as well as expanding and collapsing animations will no longer be present, fading transtitions and micro animations will still be still present.
  • None: All animations and transitions are completely removed, including fade transitions.

Links

Underline all links

Underline all links help

  • Default (Unset): Most links are underlined, with a few exceptions such as: the top level links in the main navigation (on large screens), cards, tags and icon links.
  • Yes: Will add underlines to the exceptions outlined above, resulting in every link being underlined

Text and paragraphs

Font size (main content)

Font size help

This setting does not apply to the site's header or footer regions

  • Default (Unset): Font sizes are set to site defaults
  • Selecting Large or Largest will increase the font size of the main content, the size of the increase depends on various factors such as your display size and/or zoom level. The easiest way to determine which option suits you best would be to view this text after clicking either size's button
Letter spacing

Letter spacing help

  • Default (Unset): Default letter spacing applies
  • Increased: Multiplies the font size by 0.12 and adds the sum as spacing between each character
Line height

Line height help

  • Default (Unset): all text has a minimum line height of 1.5 times the size of the text
  • Increased: all text has a line height of twice the size of the text
Line width

Line width help

  • Default (Unset): all text has a maximum line width of 80 REM units (this averages around 110 characters per line)
  • Decreased: all text has a maximum line width of 55 CH units (this averages around 80 characters per line)
Paragraph spacing

Paragraph spacing help

  • Default (Unset): The space between paragraphs is equivalent to 1.5 times the height of the paragraph's text
  • Increased: The space between paragraphs is equivalent to 2.25 times the height of the paragraph's text
Word spacing preference

Word spacing help

  • Default (Unset): No modifications to word spacing are present
  • Increased: Spaces between words are equivalent to 0.16 times the font size