043: Accessibility Audits for Large Websites with Natalie MacLees

By /

This episode is a recording of an October 2023 WordPress Accessibility Meetup where Natalie MacLees covered strategies for efficiently planning and conducting audits for large-scale sites, including WordPress sites. If you want to watch a video recording from the meetup, you may do so on the Equalize Digital website: Accessibility Audits for Large Websites: Natalie MacLees.

WordPress Accessibility Meetups take place via Zoom webinars twice a month, and anyone can attend. Learn more about WordPress Accessibility Meetup and see upcoming events.

Listen

Mentioned in This Episode

Transcript

>> CHRIS HINDS: Welcome to episode 043 of the Accessibility Craft Podcast, where we explore the art of creating accessible websites while trying out interesting craft beverages. This podcast is brought to you by the team at Equalize Digital, a WordPress accessibility company and the proud creators of the Accessibility Checker plugin.

This episode is a recording of an October 2023 WordPress Accessibility Meetup where Natalie MacLees covered strategies for efficiently planning and conducting audits for large-scale sites, including WordPress sites. WordPress Accessibility Meetups take place via Zoom webinars twice a month, and anyone can attend. For show notes, a full transcript, and additional information about meetups, go to AccessibilityCraft.com/043.

>> AMBER HINDS: I’m going to add a spotlight here so everyone can see our speaker, Natalie MacLees. I’m very excited to have you here, Natalie, so welcome. 

Natalie is a digital professional with over 25 years of experience building on the internet, and has had the privilege of working at Penn State University, where she gave a deep understanding of the importance of building websites to be accessible to everyone, and we’re really excited to have you here tonight speaking about how you handle accessibility on very large websites, because as I can imagine, you probably have like 500 websites at Penn State University, [chuckles] with many, many pages, so we’re very excited to hear more from you, and welcome. 

>> NATALIE MACLEES: Thank you so much. Thanks for having me. 

>> AMBER: Yes. I’m going to stop sharing so you can start sharing. While she is doing that, I do just want to note that I will be watching the Q&A module. I highly recommend, if you can, putting chats in the Q&A. I try to watch the actual webinar chat as well, but sometimes things get a little buried, so if you have questions, please put them in the Q &A. 

>> NATALIE: All right, ready? 

>> AMBER: Yes, I think so.

>> NATALIE: All right, so we’re going to talk about managing audits for big websites. 

First, nice to meet all of you. Thank you so much for having me. A little overview of what we’re going to cover tonight. First up, “How to select what to test?” Because if we have a website with 10,000 or 50,000 pages, we’re not going to test all of those. [chuckles] That’s just not practical, so we’ll talk about how we figure out what we’re going to test.

We’ll talk about how to break up a big project and manage it. We’ll talk about how to optimize your testing, and take advantage of the tools available, and we’re going to talk about how to track remediation, and retesting, because if a developer says, “I’ve fixed this,” you can’t trust them. You need to retest it again and make sure, and how do we keep track of all of that? 

I wanted to get started by just a quick reminder that audits don’t fix anything, so all and all it does is find the accessibility issues on the site that needs some attention, but it doesn’t actually change anything about the site at all, so when we’re conducting accessibility audits, we want to make sure that they are deliverable to our client or to the remediation team, is clear and actionable, even if it’s an enormous site with tens of thousands of pages. 

So let’s go ahead and dive in and get started. First, we’ll talk about how we select what to test, and I like to get started with the site map. For most WordPress websites, this should be readily available. You used to need a plugin to have a site map on a WordPress site, but I think since WordPress 5.5, site maps have been just automatically generated in a WordPress Core. If you happen to come across site that needs an audit that doesn’t have a site map for some reason, you can crawl and generate one.

Sometimes I don’t always have access to the site at this stage of the project, so I will use a third-party tool. They’re usually SEO tools, and they are meant to keep an eye on the website over time, but they do a fantastic job of just generating a site map and giving you a list of URLs, and because they’re meant for SEO, they’re usually a monthly subscription fee, so I’ll just sign up, pay it for one month, and then cancel it, just to get the site map, so that I have something that I can get started from. 

We need to identify not just URLs on pages of the site, but also PDF files that might need some attention, documents, spreadsheets, presentations, audio, video, et cetera. All of those things might need to be tested, so we need to make sure that we’re tracking all of them. 

Once we have the site map, we can get started with some automated testing. Since automated testing doesn’t actually take a lot of manual effort, we can be a lot less restrictive about what we select for automated testing. You still probably don’t want to run tests on every page of the site, just because that generates a lot of noise. 

If you have a WooCommerce site, for example, with a thousand products in it and we run automated testing on all 1000 product pages, there’s just going to be the same issues that are in the template over and over and over again, and it’s not actually that helpful, so we want to just make sure that the URLs we select for automated testing represent a good sample of the different types of templates and different types of pages on the site. 

Plus, as a rough guideline, you’d want to pick two to three times what you’d pick out for manual testing, and then you can go ahead and run the automated testing right away. There’s nothing stopping you from doing that. 

From there, you can plan out the manual testing. Now, since manual testing is time intensive, we have to plan carefully, and we have to figure out a way that we can prioritize the content that’s on a site, so let’s take a look at some ways that we can use to prioritize content on a really big site. 

First up, traffic. If we have access to analytics, we can figure out which are the most popular pages of the site, and it makes sense to prioritize accessibility fixes on the most popular pages, because those are going to impact the most number of people. 

We might also look at pages that had a large number of issues found in the automated testing, so if the automated testing finds, for some reason, a thousand issues on a page and most of the other pages only have about a dozen issues, that’s a good hint that maybe that page needs some attention, because it seems like there’s something going on there, so we can take a look at our automated reports and figure out which pages might look like they might be a little problematic and might need some manual testing to figure out what’s going on, and some attention from developers to get those issues fixed. 

Then we want to look for key templates. If you understand how a WordPress theme works, figuring out what the templates are and which content uses which templates is relatively straightforward, and we want to identify content that uses each of the different templates that are in the theme. 

Just a reminder that those templates, they can include conditional components or page sections, so we want to make sure that when we’re selecting a sample, that we’re getting all of those different conditional components. 

As an example, if I have a WooCommerce site and I sell stickers and T-shirts and books, pages with the T-shirt products on them will probably have some additional selectors for T-shirt size, T-shirt color, maybe even T-shirt style, you know, if I want a crew neck versus a V-neck, and obviously, those selectors aren’t going to be on my stickers and books, so I want to make sure that I’m selecting products that have that section, products that don’t have that section. 

The pages that present books for sale might have information on the format of the book. Is it an eBook? Is it a paperback? Is it a hardback? It might have the number of pages. It might have the publisher’s name. 

So there might be a section with metadata about books that obviously wouldn’t show up on the T-shirts and the stickers, for example, so I’d want to make sure when I’m selecting specific products, specific URLs to do manual testing, that I’m getting a good sample, and I’d want to have a couple of pages with the T-shirt selectors, a couple of pages with the book selectors, and then a couple of pages of the stickers that had neither of those, and maybe had a different section altogether, so just to ensure that when you pick the individual pages, you’re getting a good sample of those conditional sections or components. 

Finally, the importance of the content, and what do I mean by important content? We’ll dig into that a little bit. First up is the homepage. I can’t really think of an example where you’d run an audit on a site and not want to include the homepage, unless you were doing rounds of audits and you’d already done the homepage in an earlier round, but the homepage is a really important page. I know on websites I’ve managed over the years, it’s 40% to 50% of people who come to the site that land on the homepage, so it’s a pretty important page of a site. 

Next, we want to think about navigation, which includes not just navigation that might be in our header and our footer or sidebar, but also different parts or pages of the site that relate to navigation.

As an example, our site map, we want to make sure that that’s accessible. If we have “search” functionality, we want to make sure the search form itself is accessible, and that the search result listings are also accessible and correctly marked up. 

Our accessibility statement needs to be accessible, because everyone should be able to access that information, and we don’t want to have any barriers on our accessibility-statement page. 

Any kind of contact information, so if we are sharing our address, phone number, email address, other links for other pages that the company might own, or if we have a contact form, we need to make sure that all of those things are accessible. Those are all very important content. 

If you have any sections of the site that tell users how to use the site or are guides or help articles on how to do different things on the site, you want to make sure that those are accessible, because everybody should get access to those instructions. 

Any legal pages you have, so if you have terms of service, privacy policy, pages like that, you’ll want to make sure that those are accessible. They’re high priority. 

Finally, if you have any pages with settings and preferences. By default, out of the box, the subscriber role in WordPress does get access to the WordPress Admin to their own profile page, where they can change their password and maybe some other fields, depending on what you have going on on the site, and the WordPress development team does a pretty good job with the accessibility team to make sure that is accessible, but if you have WooCommerce or a membership plugin or some other kinds of plugins, that settings page gets moved to the front-end of the site, and you’ll just want to make sure that once it has moved there, that it is accessible, and everybody can access it to change their password, pick if they want dark mode or light mode on the site, whatever preferences that you have that people can set. Those are a high priority for accessibility. 

All right, and then we want to get ready to do functional testing. Just like manual testing, functional testing can be really time intensive, so we want to plan carefully, and what kinds of things are we testing with functional testing? This is any kind of interactive tools or transactions on the website, so the kinds of things we want to take a look at are, if there’s a cart or checkout functionality. If you can book appointments. If there are modals, we want to make sure that those are managing focus correctly and make the “close” with the “escape” key, et cetera. 

If there are carousels, those are a little bit challenging, for screen reader users in particular, so we want to pay careful attention to any carousels on the site. If there’s tabs, accordion, or other types of interactive elements, we want to make sure that we test those. 

Any kind of forms, so we already talked about the contact form, but if you have forms for people to sign up for things or to request features or anything like that, you want to test all of your forms to make sure those are all accessible. 

Of course, login and log out. Everybody needs to be able to do that, and commenting should be accessible. 

I will note that there are plugins that you can get for your site that allow you to favorite comments, right? That’s not available out-of-the-box. Or favorite posts, and very few of those are accessible, so be careful if you add that functionality. Carefully select your plugin for that. 

Then any kind of embedded content, so if you’ve embedded a social-media feed, if you’ve embedded videos from YouTube or Vimeo, for example, if you’ve embedded forms from a third party that aren’t built in your WordPress Admin. All of those kinds of things, you would want to make sure that you test to be sure that they’re accessible. 

All right. Let’s talk about how we break up big projects, because I’ll tell you, what you don’t want to do is spend weeks or months doing a huge accessibility audit on 100 pages of a site, and turn it over, you know, turn over this list of 10,000 accessibility issues you found and find out that a good chunk of those are no longer valid because content has changed on the site, and to find out that the company has only assigned one developer to the remediation project, so we want to make sure that when we’re turning over the results of our audit, that they’re relevant, that they’re actionable. 

So we want to think in sprints. The agile methodology is really helpful here, and we always think a sprint is going to go like this: that we do the audit, and then the developers remediate the issues that we found, and then everything is fixed, but the reality of how that usually goes is, you finds some issues, the developers remediate, test again, and find new issues. [chuckles] Or the fixes weren’t complete, or the fixes weren’t the correct fix for the problem, so it does tend to be a little bit of a cycle, so it’s best to be prepared for that and plan for that ahead of time. 

Some questions to ask yourself as you’re thinking about breaking a big project into sprints. First, “How big is the remediation team?” So it’s good to know that up front, because if you only have one part-time developer who’s going to be working on remediation, you don’t want to overwhelm them and you don’t want to send them three years worth of work, because it’s probably not going to get done. It’s just going to be overwhelming, so have a good idea of who is going to be doing the remediation work and how many people there will be involved. 

Then you want to know how much accessibility experience the remediation team has, because if it’s a team that’s relatively new to accessibility, they’re going to need a lot of coaching and a lot of hand-holding in order to be able to resolve the issues and fully understand them, and that’s a lot of drain on your own time, so you don’t want to overwhelm yourself and have no time left to do additional testing.

You want to find out how often the content on the website changes, so if there’s an SEO team that’s constantly playing around with different landing pages and funnels and things like that, it’s important to know that up front when you’re planning which URLs will be involved in the testing. You don’t want to waste time testing, like, a landing page that’s just up for a couple of days as a test, because even if you find issues, there’s no time to remediate it before it gets taken back offline again.

Finally, you want to look for things that you can shift left, and what this means is things that you can move earlier in the process so that they’re not hanging out there. 

In the example I gave earlier where we have an SEO team who’s constantly testing different landing pages and funnels, are they consistently producing inaccessible pages with lots of issues on them? And if they are, what could we do to fix that situation? Could we provide a different tool for that team to use to build those pages? Could we provide some training for that team so that they would understand how to build things more accessibly? What can we move so that we’re not just doing a bunch of accessibility testing kind of at the very end of the process once new pages are live on the website?

>> STEVE JONES: This episode of Accessibility Craft is sponsored by Equalize Digital Accessibility Checker, the WordPress plugin that helps you find accessibility problems before you hit publish. 

A WordPress native tool, Accessibility Checker provides reports directly on the post edit screen. Reports are comprehensive enough for an accessibility professional or developer, but easy enough for a content creator to understand. 

Accessibility Checker is an ideal tool to audit existing WordPress websites find, accessibility problems during new builds, or monitor accessibility and remind content creators of accessibility best practices on an ongoing basis. Scans run on your server, so there are no per page fees or external API connections. GDPR and privacy compliant, real time accessibility scanning. 

Scan unlimited posts and pages with Accessibility Checker free. Upgrade to a paid version of Accessibility Checker to scan custom post types and password protected sites, view site wide open issue reports and more.

Download Accessibility Checker free today at equalizedigital.com/accessibility-checker. Use coupon code accessibilitycraft to save 10% on any paid plan.

>> NATALIE: For big websites, it’s important that we prevent new issues from happening. It’s just as important as making sure that we take care of the issues that are there. 

>> AMBER: Can I interrupt for one second? Because there was a timely question. Could you explain what “shift left” means?

>> NATALIE: Oh, sure. Yes. I guess it’s very relevant if your native language is a left-to-right language. [chuckles] So you’d normally start on the left and go to the right if you’re listing out your process, right? So you’re going to do design, and then you might test the design, and then you might do development, and do some testing on the development. 

What we mean when we say “shift left” is that, often, accessibility testing gets left for the very last step, all the way on the right. The very last thing over there. “Oh, yes, we should test this for accessibility,” and what we mean by shifting accessibility left is putting it earlier in the process. Can we think about accessibility in the design phase? Can the designer be thinking about color contrast, and usability, and different ways that we can accommodate people with cognitive disabilities in their design? Can we incorporate some kind of automated accessibility testing for developers so that every time they commit code, there’s some kind of automated accessibility testing running on that code? Can we run accessibility tests on a staging site or a development site, manual tests there to make sure that things are accessible before they are released publicly? That’s what we mean when we say “to shift left” on accessibility.

All right. Let’s take a look at some examples of sprints. I find that I’m often working with a remediation team who doesn’t have a lot of experience in accessibility, as clients will often might use their internal developers for, you know, when they have a large website, they have, usually, an internal development team and they just want that team to take care of remediation, but that team doesn’t necessarily have a lot of experience with accessibility. 

So we might, as a first sprint, just do the homepage, especially if it’s a long or complicated page. That might be one sprint just on its own. We could go through any kind of contact, so if there’s contact information in the footer, a separate contact page, maybe there’s multiple contact pages. For a big company, there might be a contact page for each of their department, so we might have a sprint for accessibility where we just test and remediate issues specifically related to contact information and forms. 

We might check the cart and checkout flow. Very important for an E-commerce site to make sure that’s all accessible. We might have a sprint where we just focus on navigation, just product pages on an E-commerce site. We might do just landing pages, so like we talked about earlier, an SEO team creating landing pages. We might do one where we just focus on blog posts, and we might do one where we just focus on the legal pages. 

So we have lots of different ways that we could break a big project up into really small, manageable pieces that can be done in as little as a week or two. 

Next, we want to figure out how we can optimize our testing and take advantage of all the tools that are available to us to make the process go as quickly and smoothly as possible, so we’ve got three different types of testing that we do as part of an audit. We have automated testing, manual testing, functional testing. 

We all know automated testing is limited, right? It can only find 25% to 30% of the types of accessibility issues that can be found on a site, but automated testing can also offer some really helpful guidance for our manual and functional testing, and that can help us to save a little bit of time in those types of testing. 

So an automated scan can identify images that are missing alt text, but of course, what still needs manual checking is, is that alt text relevant and accurate? 

Automated testers can check if our form fields have labels, but they can’t check, “Are those labels helpful and descriptive?” And they can’t check, “Did we use some other method of applying a label to a form field?” For example, did we use the Aria label attribute on the element? 

So we would want to make sure anything that gets flagged with automated testing.., sometimes those are just things that we need to check manually, and not actually issues. 

All right, so we run the automated test first and we use those to help us identify problematic areas of the site or problematic components. Then we want to be sure that we’re recording all issues in one spot, so we want to make sure that our deliverable to the client, to the remediation team at the end of the audit is something that is easy to understand and actionable, and one way we do that is by having all of the issues together in one spot. We also want to identify the issues by severity and priority, so severity, I think we’re probably all familiar with. Critical issues are ones that are blocking one or more groups from accessing some page or part of the site, and then we have serious, moderate, lower-priority issues.

Now, you would think that the more severe issues are always going to be the highest priority and that is the case most of the time, but it’s not the case all of the time. 

I’ll give you an example. If we have a serious issue, but it only occurs once on the site, and maybe it occurs on a page of the site that doesn’t have very much traffic, but then we have another issue that’s moderate, but it occurs 100 times on the site, and it occurs on pages that get a lot of traffic, we might actually decide that the moderate issue is a higher-priority fix than the serious issue, so when we know how often an issue happens, when we know how many pages of the site are affected, that helps us make some decisions to prioritize fixes. 

All right, and then we need to make sure that we have a way to track remediation and retesting, because developers are going to work on the issues that have been found in the audit, and sometimes, the first try, they’ll get it right and the issue will be fixed; but other times, that’s not the case, so we want to have ways that we can track through the process, and make sure that we have a way to track when we have retested something and it is not meeting the accessibility guidelines and we need to send it back to the developer for more work. 

Know that we often record accessibility issues in a spreadsheet, but a spreadsheet can be kind of hard to track. We could have a column for who’s assigned to it, we could have a column for the status, but it’s really hard to have a conversation about the issue. It’s hard to go back and forth with the developer a few times in a way that’s tracked in the spreadsheet to figure out what should be done to correct a certain issue or to help the developer understand the issue, but we do want to make sure that we get everything in one place, because the results of our audit, they have to be manageable and not overwhelming, if anything’s going to get done, so if we turn over something that’s confusing or overwhelming, the chances of the remediation getting completed are pretty low. 

So we want to find a way to take all of our automated testing, our manual testing, and our functional testing and get that all into one place where the issues and statuses can be tracked and the assignments can be tracked, so manual and functional issues could be recorded directly in an issue tracker, or they could be imported from a spreadsheet. Automated tests are usually going to be a CSV. They might be like a JSON or something like that, but hopefully, whatever tool you’re using will allow you to import those kinds of issues and get them all into one place where they can be easily tracked. Manual issues could go directly into software, could go into a spreadsheet, and then imported later. Automated issues could be imported from whatever tool was used to run them. 

If you’ll indulge me for just like two minutes, I’d like to share a little bit about a tool that my team and I have been working on, AAArdvark. We’ll be launching soon, and it is a tool for helping you to manage accessibility audits. It does include automated, manual, and functional testing all in one spot. All of the issues reported in one area. You can select an element in the UI of the website and report an issue with it right there. 

Once you pick what the issue is, it will automatically figure out which WCAG guideline that applies to or success criterion that applies to and present you with some recommended fixes and additional information about it, so you don’t have to type that information in by hand. 

You can also see directly on the website where both automated and manual issues are, and it will allow you to assign issues, track their status, have a common thread on each issue, and report on your progress over time. 

We are launching soon. It’s AAAardvark Accessibility with three A’s, if you’d like to check it out. “AAArdvarkAccessibility.com.”

All right, thank you for indulging me just for a second on that. I just wanted to share about it. OK, so quick review of what we’ve talked about. We have four big challenges when we’re doing audits for big websites. 

First, we need to select what we’re going to test, and that could depend on traffic. It could depend on the number of automatic issues that were found on the different pages. We want to make sure that we get key templates and include any conditional components or sections of those templates that might be visible or not on different pages, and then we want to think about the importance of different pages and ensure that we’re prioritizing the important content on the site. 

Second, we want to break a big audit into sprints. We don’t want to deliver this enormous report at the end of a… You know, weeks or months-long audit to the client to try to remediate. We want to break it up into work that can be done in a week or two at a time so that we’re not overwhelming anybody and we’re making good progress on the website, and also, we don’t have issues that are months old that have been sitting there becoming obsolete. 

We want to take advantage of automated testing and let that guide us along, and really integrate our automated, manual, and functional testing all together so that each type of testing supports the other two.

Then we want to make sure that we are delivering a report to the client and the remediation team at the end of the audit that is actionable, that’s easy to understand, that is easy to track and assign the issues, and get all of those in one place so that they’re really easy to understand.

OK, are there any other questions? 

>> AMBER: Sorry. It always takes me a second to get myself back up. [chuckles] 

>> NATALIE: No worries. 

>> AMBER: Someone said they love the name, they’re excited about that, so we’ll all have to go check out AAArdvark Accessibility. 

There were some questions that were asking specifically around what other tools you use for automated testing, so do you want to talk about some other tools that you recommend, or that you find helpful? 

>> NATALIE: Sure. I really like the Axe tool from Deque. I find that one really helpful. I like that it’s integrated right into the browser. 

I also like WAVE from WebAim. That tool is super helpful. I think it’s really nice also just as a site owner that you don’t have to install anything, that you can just go to their website and run the test and see the results right on their site, and not have to install a browser extension, you don’t have to do anything else. 

I do like ANDI, that’s from the Social Security Administration that gets used in Section 508 testing. It’s a bookmarklet. It’ll help you along in a lot of manual tests, but it does some automated testing as well that I actually really like. The interface isn’t super pretty because, you know, it’s made by the government, but it is actually really helpful. 

>> AMBER: Great. Thank you. Beth was asking: “Will AAArdvark have API endpoints?” 

>> NATALIE: Yes, it will. Yes. [crosstalk ]

>> AMBER: Are those going to be there when it launches? Can you give us a preview? 

[laughter] 

>> NATALIE: They will be there when it launches. Can I give you a preview? I might be able to give you a preview. Let me see how our beta site is looking. [laughs]

>> AMBER: Everyone loves live demos. [laughs] 

>> NATALIE: I know. 

[crosstalk ] 

[laughter] 

Let me see. Trying to find a good browser window here. All right. Let me just take a peek at our beta site, make sure it’s not down or anything. I want to make sure that I can log into it. 

I can answer another question while I’m doing this. 

>> AMBER: OK. Simon asked: “Is it also useful to record effort to fix in addition to severity and priority?” Do you have any opinions on that? 

>> NATALIE: Yes, you definitely could. You might decide that that’s one of the things that goes into determining your priority. You might make that part of your priority calculation, or you might track it separately. It’s up to you. 

Some issues are super quick, like, a five-minute fix, and others are quite complicated, and might be a full day or even multiple days, so you might want to track that just to make it helpful, and maybe tackle all the quick issues first for some quick wins, and then come back later and plan out, you know, maybe a sprint just to fix a bigger accessibility issue that might have some issues. 

>> AMBER: Yes. In my experience, sometimes it’s hard with that, you know, the effort level, because we often do it for entities that are not ourselves, right? If it’s your own team, it’s easy to know what the effort level is, but sometimes you think it’s low effort, and that person thinks it’s really high effort. 

An example that came up on an audit for a community college, and we were doing audit and remediation, and we’re doing the hard stuff, like the code stuff, and we flagged that there were quite a few posts where they weren’t using the WordPress list block, and they were just typing it out with, like, a hyphen, or just typing the number one, and then hit “return.” So they were all in paragraphs and they weren’t actually in real lists, and we were just like, “Oh, this is a quick fix,” and then it came up that the content person literally didn’t know how to put things in a list and felt really overwhelmed when we were, like, “We’ll just put it on the list.” [chuckles] Right? 

>> NATALIE: Right. 

>> AMBER: And so I feel like that’s the one thing that is challenging about that effort, which is everyone’s effort is very different. 

>> NATALIE: Yes, and if they’ve done that once, it’s just a few minutes to go fix it, but if they’ve written a hundred posts like that, [chuckles] it’s going to be harder to go back and fix. 

>> AMBER: Yes. Do you want another question? Or are you… 

>> NATALIE: Yes. Our beta is not in super great shape right now. I don’t have the right plugin installed on my test sites, and I don’t think I have anything super impressive to show. [laughs] 

>> AMBER: That’s OK. That’s OK. 

[laughter]

I know I put you on the spot. 

[laughter] 

Maureen was wondering if you could define what a functional issue is? 

>> NATALIE: Oh, sure. The functional testing is testing through, like, a user flow, or a user story. Like, a user is going to put an item in their cart, and then they’re going to go to the “view cart” screen and then they’re going to go to the checkout screen and enter their payment information, and then they get a confirmation screen at the end, so that would be like a functional user flow, and we would want to test that from end to end. Can you, without using the mouse, get through that flow? Can you put an item in your cart? Can you get to the cart page? Can you enter your credit card and all of your payment and shipping information and get through to the confirmation page? Can you do that using a screen reader? 

So a functionally issue would be that there’s some kind of problem in that flow, that there’s something that’s blocked or something that you’re not able to do with some kind of assistive technology or on certain different devices. 

>> AMBER: Yes. Thank you. 

>> NATALIE: Sure.

>> AMBER: Simon asked: “What do you do when sprints overlap with each other due to time constraints?”

>> NATALIE: Well, that is the most fun of all. 

[laughter] 

So in that case, you have to be mindful of any individual on the team. Are they involved in both sprints? Because if they are, you have to think about how you’re planning out work for them. They can’t get a full 40-hour work week on both sprints. [laughs] So you just have to kind of be mindful of who’s involved, and how much of their time is already taken up by the first sprint that’s wrapping up, and how much time will they realistically have left to work on the second sprint, and it is a balancing act to kind of try to figure out, like, should they spend half a week on each sprint? Should they just focus on that first one and get it out the door? And it’s going to depend on what the priorities are, on what the overall deadline is, you know, which way it goes, and sometimes when people have vacation schedule, [laughs] it’s going to come into it as well. 

>> AMBER: Yes. 

>> NATALIE: Yes, it’s a good balancing act and it’s a good exercise in figuring out how to pay attention to multiple priorities at the same time, and as with most things to do with web development and accessibility, the only real answer is always, “It depends.” “It depends on the situation.” 

>> AMBER: Yes. I think we’ve found the same thing. If you only have one developer, it’s really hard, I think, to run multiple sprints at the same time, and so that’s really where I would say, if there’s only one person focused on fixing it, even if it’s not a developer, only one content person, you really do just have to prioritize at which point is, like, “Let’s just do this, and then when we’re done, do that,” and if this takes longer, it makes more sense to just push this down on the calendar instead of trying to start something new when you haven’t even finished the first thing. 

>> NATALIE: Yes, yes. It can be really hard to do that, and some people get really overwhelmed by that, and don’t work well in that situation either. [chuckles] 

>> AMBER: Yes. Lucy Candice [phonetic] said: “I’m struggling with how to quote a site audit without a ton of work.”

I’m guessing she might mean, like, spending a ton of time digging into how many problems already exist? Do you have any thoughts on that? Like, how you approach quoting them, or scoping, and even figuring out what the job is? 

>> NATALIE: Yes. I know that can be really hard, because you feel like you have to do half the audit to figure out what the scope of the work is for the audit, and of course, you don’t want to have to work for the audit before you’ve even been paid. 

I do find if you can talk to the client and get them to agree to maybe a smaller sprint or a series of smaller sprints that you could quote more accurately with a lot less work, that’s super helpful. If they won’t agree to that and they just want to know how much the whole thing is, sometimes you just have to take the best sample you can kind of grab without a lot of research and thought into it, run the automated testing on it to get a feel for, “Wow, how is this looking?” Is this above-average number of issues? Below-average number of issues? 

Often, automated testing is an indication of how much more you’re going to find in a manual, and then you can go from there, but I think I’ve always had to quote, like, a range rather than a single price for what I think an audit is going to cost. Like, “It’s going to be probably at least this much, but no more than this much,” and it is really challenging to do without tons of work, and it still does take at least a couple of hours to come up with a reasonably accurate quote. 

>> AMBER: Yes. We actually gave a talk earlier in the year. You can find it if you go to the recordings for anyone’s… Or maybe Paola can stick the link in there … About how we’ve been approaching. We do, like, audit, remediation, retainer-based projects. Mostly because we want to get away from what you were saying, which is an audit doesn’t fix anything. [chuckles] And so we’re, like, “Let’s find a few problems and fix them, and find a few problems and fix them.” Like that kind of thing. 

I think we have the same problem, especially when you’re getting into remediation, where people are, like, “We’ll, how long will it take?” And it’s, like, “Well, I haven’t even audited your whole website, so telling you how long it’ll take to fix the whole thing is a little hard.” 

Although, I do think what we have figured out is that we can give them a minimum idea, even just like by what you’re saying, like, we’ll typically use WAVE, and we have a modified version of our Accessibility Checker that allows us to scan a website that it’s not installed on, so we don’t even have to get access to their site to scan with our tool, and then we will do, like, a keyboard, like, we’ll just tab through the homepage and the nav menu, because that right there, like, if the whole navigation has to be rebuilt on a WordPress website, depending on how many hours they can budget in a month, that could be all their time just doing the nav menu.

>> NATALIE: Yes. Yes. 

>> AMBER: So in that talk, we ended up being, like, “OK, we’re going to put minimum month commitments on these, because we know it’s not going to be less than this number of hours.” 

I feel like if you’re just open with your clients about, you know, “Until I get in, I’m not really going to know, but the goal of the first month is that we’re going to have a better picture of what the subsequent months will look like”?

>> NATALIE: Of where we’re going, yes. 

>> AMBER: Yes, and it’s like paid discovery for a website design project, right? Like, your clients have to pay you to do discovery before you can really fully finalize the quote for the whole build. I think it’s kind of the same thing. 

>> NATALIE: Yes, and I think it’s helpful, too, to set the expectation with the client that accessibility is not, “We’re just going to do this one time and then we’re done forever.” [laughs]

>> AMBER: Yes. 

>> NATALIE: This is, like, as long as you’re working on the website and making new content, you need to keep thinking about this. 

>> AMBER: Yes. That was part of what motivated us to set these up, too, is like, there are subscription. People literally subscribed and it auto bills them every month. Now, some of the bigger institutions, they don’t pay with a credit card, right? Like, it’s still a PO, but it hits the PO and it’s the same price, but it’s the idea of trying to build this mindset that accessibility, it’s not, like, “Make it accessible and you’re all done.” But it’s something you should invest in, the same way you invest in SEO or web design or something like that. 

>> NATALIE: Yes. 

>> AMBER: Hopefully, that’s a helpful answer, Candice. If not, let me know. I did see she said in the chat she’s glad she’s not the only struggling with quoting on this. I think quoting can be very challenging on many fronts. [chuckles] 

>> NATALIE: Yes. 

>> AMBER: You know, [crosstalk ] 

Yes, so we had an anonymous question that I don’t know if you know the answer to this one. It’s specific to the Contact Form 7 plugin. Do you know much about that plugin? 

>> NATALIE: I know more about it than I wish I knew. [laughs] 

>> AMBER: OK, awesome, so this person says, “Is there a way to apply Aria labels on Contact Form 7? 

>> NATALIE: Oh, that’s a good question. I actually don’t know the answer to that off the top of my head, so I [crosstalk ] feel like I maybe have, but I’m not 100% sure. I’m sorry. Go ahead. 

>> AMBER: Well, what I was going to say was, I haven’t used Contact Form 7 for many years, but when I last looked at it, I thought that it gave you a field where you select things and then it spits out HTML?

>> NATALIE: Yes. 

>> AMBER: And so if that is the way it still works, I think the answer is yes, because you can just type in the HTML. 

I would advise you to pick a different form plugin. 

>> NATALIE: I would too, but if you must use Contact Form 7, there is a plugin called Accessible Defaults that you have to install before you make any forms in Contact Form 7. [laughs] It’s like accessible [crosstalk ] defaults for Contact Form 7.

Yes. It’s super nice, but if you’ve already got your form set up, it’s not that helpful. You have to install it before you make any forms, but it is helpful for getting your form to be accessible by default, because you do have to do a good amount of coding work on a Contact Form 7 form to make it accessible. 

>> AMBER: I was trying to see if I could find it real quick. I found the “WordPress.com” page, but I wanted to find the WordPress… Oh, wait, if I click on that now, I should be able to get it too. 

Nope, somewhere there’s the… There we go. I can put that in the chat for anyone who is interested. 

>> NATALIE: Yes. 

>> AMBER: There we go. OK. 

So Eagle [phonetic] asked about links to different tools. Do you want to put your AAArdvark link in the chat for people? We’ll make sure that we get those in the notes as well. 

There have been a few that Paola has been putting in. I know that she did put in a link to Axe and ANDI, and WAVE is just “Wave.webaim.org,” but we can also put that in the chat as well. 

Gerson, I noticed, just commented that Accessible Defaults has not been updated for two years. I think it still works. 

>> NATALIE: Yes. In my experience, it still works. 

>> AMBER: Sometimes they don’t actually need to be updated. [chuckles] And this one’s from Joe Dolson, who is on the WordPress Core Accessibility Team, so I would assume that it is good. He wouldn’t let it languish if it wasn’t.

>> NATALIE: Yes. 

>> AMBER: Simon had a follow-up on the effort to “fix” question: “Do you have a formula to calculate priority? Or is there some subjectivity to assigning priority?” 

>> NATALIE: My answer to that is both, actually. I’ve been working out the formula that we’re going to use in AAArdvark, and figuring out how to take the severity and the number of pages the issue occurs on, and then in the future, an integration with Google Analytics that’ll let us get traffic volume for different pages, and to kind of take all of those things into account and have a formula that will calculate a priority, but of course, there’s always exceptions to what you can calculate automatically, and there might be an upcoming promotion or something that’s running that’s going to bump a certain page or section of the site up in priority, where you would have to manually go in and change it. 

So you can do a formula to calculate it based on whatever things are important to you, time to fix, severity, number of occurrences, et cetera, but then you’re going to want some wiggle room in that to adjust that as needed for different situations that the formula obviously doesn’t know about. 

>> AMBER: Yes. I know for us, like on the severity range, we… I’ve seen some companies where they do have, like, five different severities, and I’m, like, it’s too many.

Critical, like, really high is, it’s breaking, like, there’s no ability for someone with disabilities, and we have medium, which is, it’s not breaking. There’s maybe obvious workarounds or it’s a really minor piece, but it’s still technically a WCAG failure and therefore might file Section 508, or AODA in Ontario, or something like that. 

Then we have low, which is best practice, but not clearly a failure of anything, and that’s all we do. It’s either it breaks something or it doesn’t, [chuckles] you know? 

>> NATALIE: Yes. [laughs] That’s true. [laughs] 

>> AMBER: Yes, but I like what you’re saying about trying to mathematically calculate in with, like, analytics page views to save the site owner from even having to think about that. 

>> NATALIE: Yes, because you automatically know now which are the pages with the most traffic; and that, of course, changes over time, so you want to keep up with it. 

>> AMBER: Yes. There were some comments and questions about the countdown timer on the AAArdvark website says 15 days until launch. 

>> NATALIE: Yes. 

>> AMBER: You have a captive audience right now. Everyone’s really excited, [laughter] which is fun, because I didn’t even know, so I’m, like, “This is neat too.” Is that like a beta? What’s coming in 15 days?

>> NATALIE: Yes. Yes, like a beta. I guess we could call it our minimum viable product. It’s what we’ll be releasing in 15 days, and we are planning to run a promotion at launch, where you could get a free account for one website, if you want to sign up and take advantage of that, but yes, we are looking at launching by November 7th or so. Yes. 

>> AMBER: Very cool and exciting, someone else said: “I struggle to find accessibility instructions for a specific components, like how voiceover should read a carousel, or how to define Aria label for specific components. Do you have any good documentation sites or anything that tackles best practices for UI components?”

>> NATALIE: Yes, I do, actually. Haydon Pickering has a website and eBook called “Inclusive Components,” that has some really nice skeleton code of all different types of components and how they should be set up, and which attributes should be there, and how they should be read out by screen reader, and it kind of talks you through and lists all those things out, so that’s one of my favorite ones. 

>> AMBER: And that’s “InclusiveComponents.design,” right? 

>> NATALIE: Oh, yes. Dot design, yes. I knew it had a weird TLD. [laughs] Thank you, so that’s been a really helpful resource. 

Then the W3C actually has a lot of pretty decent examples, actually, of different types of components and how they should be marked up, and they have, like, this magic link that will open it in a CodePen for you, so you can get a working example and kind of play with it and try changing things and then have a screen reader walk through it, so I’ve actually found that to be super helpful, and it’s just part of the same website that the WCAG guidelines are on. 

>> AMBER: Yes. The other one that we look at a lot is the USWDS. Do you use that? [crosstalk ] 

>> NATALIE: No, I haven’t used that one before. 

>> AMBER: Let me see if I can pull it up real quick and put it in. It’s, “DesignSystem.digital.gov.” So this is the design system for federal websites in the United States. It doesn’t have everything, like, they don’t have carousels, but that’s because they don’t allow carousels to be used on their websites, but it has accordions, and it has, like, cards, and other kind of components, so that’s one that we’ve referenced quite a bit, and if you are building a website that’s federally funded, then you have to use that.

>> NATALIE: Yes. [laughs] 

>> AMBER: I think I saw that there was one in the UK also, but I’m forgetting what the web address is for that. 

>> NATALIE: Yes, you’re right. I have seen it and I don’t remember what it is either. It’s the same thing, the design library for government websites. 

>> AMBER: In the UK, yes. 

Gary [phonetic] said: “Do you have any advice in motivating open-source tools, tool teams in implementing accessibility recommendations?”

>> NATALIE: Oh, goodness. [laughs] So yes. 

>> AMBER: Call them out on social media. I don’t know. 

[laughter]

No, no, no. You probably have a much better answer than that. 

[laughter] 

>> NATALIE: So I have been involved with the WordPress Accessibility Team, although for the last year, I’ve been working really hard on AAArdvark and haven’t had a lot of time, but I have been involved with the accessibility team for WordPress for quite some time, and it can be really challenging to have your voice heard, and different communities are more and less open to it, so it kind of depends on which open-source product you’re talking about as well. 

I have found that giving talks at conferences has been really helpful to get some attention and to help people understand what the issue is. I found, a lot of times with developers, a lot of developers make an assumption that people who can’t access the websites that they’ve built, for whatever reason, just need better tools, like, they just need better assistive technology, and that there’s nothing for the developer to actually do, so to help them understand how the choices they’re making in their development process are impacting other people and how they could make different choices that don’t actually take that much extra time and have a different outcome I think has been really helpful and to start to win people over that way.

Then just being active in the communities and asking for it over and over again, and I know it feels like it’s not working and you’re not getting anywhere, but it does actually work over time, and I think WordPress is more accessible at this point. I know it’s far from perfect, but I think it’s more accessible at this point than it would have been if the accessibility team wasn’t there, advocating and making presentations and getting attention for the cause.

>> AMBER: Yes. I think one of the things that has really helped with some of the plugins in WordPress… It’s really easy for those of us that know accessibility to try out a plugin and be, like, “Oh, it’s not accessible,” and just abandon it and never communicate that to anyone, because we’re just moving fast. We’re looking for something, we’ll be like, “Oh, this doesn’t work. I’m not going to use it.” Because it does take time to report things, but honestly, this year, that was one of my goals, which was that I was going to try and provide more feedback to plugin developers, because a lot of times, they just don’t know. 

>> NATALIE: Yes, they don’t know. They have no idea. 

>> AMBER: I have found that I’m getting positive response, but I think if you want to see that positive response and that motivation, it’s really helpful that you write really good… Like, if it’s on the support forums on “WordPress.org” or on a public GitHub repo, you have to really write a detailed explanation of what the problem is and how to fix it. 

So some of these component libraries that we talked about, you know, if I am trying to plugin and it has an accordion that’s not accessible, I’ll be, like, “Your thing is missing this kind of Aria,” and I’ll literally list out, you know, like, “It doesn’t have Aria-expanded on it.” Or, “It’s not using buttons,” and I’ll list out what all the problems are, and then I’ll say, “Here’s where you can go see an example,” and I’ll link them over to the USWDS.

I found that putting that extra time in, if I can… Like, I try to do it, at least one or two a week, just to try and get it out there, but if you give a really good feedback, I think most developers are open to it and will respond positively.

I’ve seen somewhere, where we flagged something and the developer fixed it, like, within 24 hours, so it’s just amazing to me when it’s a free plugin on “WordPress.org” that they’re not getting paid for. 

>> NATALIE: I’ve had that experience too, where they fixed it, like, right away, and it was super nice. 

I think on the flip side of that, too, is to leave positive reviews for the people who are accessible. [laughs] And so I try to do that, too, to just say, “Hey, thank you so much for building an accessible product. I really appreciate it.” [laughs] 

>> AMBER: Yes. I do that. I’ll leave them a five star review. Or the other thing is, for some of the plugins we use all the time, if I monitor their change logs or I’m on their email list, so I get notified when they’re updating, like Advanced Custom Fields, for example, because I’m like, “I want to know what’s happening in that, because we use it on all of our websites..,” and if I notice, in their change log, that they did an accessibility thing, I’ll frequently go on Twitter and I’ll tweet and tag them and be, like, “Hey, shout out to Advanced Custom Fields for making their settings page function with a keyboard.” That was a big thing that they did earlier this year. 

I think positive reinforcement works, right? We talk about it with our children too. If all we do is shout at the people who are doing a bad job, I think it gives accessibility this negative kind of persona, and so it’s good to have a positive on it too, which is, like, “Hey, you’re doing a good job.” “Look at this other company.” You know, and then maybe that plugin’s competitors will be, like, “Well, crap, I’m not getting awesome shout-outs.” [laughter] So I need to do this too, right? 

>> NATALIE: Yes. Fingers crossed. 

>> AMBER: That’s the hope. 

Miranda [phonetic] said: “Is there a good sample summary report I can reference?” 

Miranda has logged accessibility bugs, but has never summarized it or written a report. 

>> NATALIE: Oh, interesting. I can’t think of one off the top of my head besides the horrible ACR/VPAT, which is not helpful to anyone. [inaudible]. 

>> AMBER: Well, I don’t know if those are horrible.

[laughter] 

For anyone who’s not familiar, can you describe what an ACR, Accessibility Conformance Report, and a VPAT is? And then maybe we can throw a link?

>> NATALIE: Yes. VPAT is Voluntary Product Accessibility Template. It’s the template you use to produce a report called an ACR, an Accessibility Conformance Report? Is the “C” conformance or compliance? [laughs] 

>> AMBER: I think it’s “conformance.” 

>> NATALIE: Yes, so it’s a standard format for reporting the state of accessibility in a product, which could be a website, but it could also be a desktop software, like Microsoft Word, and it just has you step through kind of one WCAG guideline or success criterion at a time and says if the product conform, not conform, or conform with exceptions. [laughs] And then you have to list out what the exceptions are. 

>> AMBER: Yes. Paola put two links on the Section 508 website that explains to it. I’ll put a link to the VPAT for Accessibility Checker, so it’s a software; it’s not for a website, but if anyone wants to look at that format, you can. 

The other thing that stood out to me, and I’m trying to see if I can find it real quick, as a good example of a written report that’s not an ACR was the Tenons [phonetic], which Tenon doesn’t exist anymore because they got purchased and kind of went away, but Tenon and Carl Groves, they were commissioned to do an accessibility audit of Gutenberg, and that report, everything in it has pretty much been fixed, but that report was more of a narrative style, and I think I can find it. Just giving me one second. [crosstalk ] another example. 

>> NATALIE: Yes, that’s a good one. 

>> AMBER: I’m on the blog post. Oh, here it is. OK, so this blog post on the WPCampus website? So WPCampus, if people aren’t familiar, is a nonprofit organization that is a WordPress in Higher Ed organization. It’s super awesome, and there’s a lot of accessibility conversations there. They were actually the ones who commissioned this. 

So if you go down under the results of the accessibility audit heading, there is a 34-page executive summary which is a PDF, and if you click that, you can open that, and then there’s other documents here that talk about like their approach to the audit and things? So I think if you’re not familiar with writing audit reports, this actually might be a really interesting one to reference. 

The narrative style, this is Tenon’s format. It’s not like it’s an official format. [crosstalk ]. 

>> NATALIE: Yes, it’s not a standard format. 

>> AMBER: Yes. I mean, anyone can do what they want. There’s things in here that are similar to what you said, right? Like severity or that sort of stuff? So I think every company and every auditor has to decide what they want their format to be, but I think looking at this might be an interesting look, if you’ve never seen what other companies are doing, or if you’re trying to think about what your own format should be.

>> NATALIE: Yes. 

>> AMBER: I don’t know of any other public ones like this. Do you? 

>> NATALIE: Yes, I know. That’s why I was, like, “I’m having a hard time thinking of one.” [crosstalk ] if Drupal doesn’t have something, but I don’t know of it specifically.

>> AMBER: Yes. The other thought that I had, which I don’t have a link to one off the top of my head, but I feel like there are some colleges that have gone under a mandated remediation with the Office of Civil Rights as part of the Department of Education, and they usually have to publish their audits because they’re public record.

>> NATALIE: Nice. 

>> AMBER: So I feel like if you’re a good Googler, you could probably find one or two of those for, like, a college website, and then that might give you some other formats, maybe. 

Gerson asked: “Is the way of calculating severity and priority…” I think this is referencing what you had previously talked about with maybe using Google Analytics and things like that. “…Is that going to be available in the AAArdvark tool?”

>> NATALIE: It will be. Not at launch, but shortly after launch. [chuckles] 

>> AMBER: I totally understand about that.

[laughter] 

>> NATALIE: We had to do a 1.0 and draw a line in the sand. “This is in 1.0. This is after.” 

>> AMBER: Yes, and then the last question that I think I have… Unless anyone else wants to add any.., and this is more of a technical question, but I’m going to answer it verbally so everyone has it. 

Doug had asked about turning on the ability to save chat in Zoom, and this is a limitation of Zoom webinars. In Meetings, participants can save, but unfortunately, in Zoom webinars, we can’t, and I’ve spent a bunch of time digging into all the Zoom settings, and it can’t be done, but all of the links that we’ve put here will be in the recap.

Paola very awesomely goes through the chat log from the recording and pulls them out and puts them in a bulleted list, so once the recap goes out, you will have all of these links, if you don’t want to have to try and frantically click them all right now.

[laughter] 

>> AMBER: I think that’s it for questions. 

Natalie, can you tell people how they can get a hold of you if they want to follow up? 

>> NATALIE: Oh, sure. Yes. Can I share my screen again real quick? 

>> AMBER: Yes. Let me know if you can’t. 

>> NATALIE: [chuckles] OK. I do have a last slide here with my email address, which is kind of a mouthful: “Natalie.maclees.nsquared.io.”

I don’t really use social media much anymore. [laughs] And then, of course, the website, “AaardvarkAccessibility.com.” 

Once it’s launched, we’ll have contact information on it. I know right now is just the countdown. 

>> AMBER: Awesome. Thank you so much. This has been really fun, and I know that you probably have at least 50 new people who have all gone to join and get notified when your tool becomes available. [laughs] 

>> NATALIE: Oh good. I’m glad. I’m excited. 

[laughter] 

All right. 

>> AMBER: Thank you everyone for attending. The recording will be available in about two weeks. 

We’re going to do the slightly awkward thing where we say goodbye, but we sit for a minute, and I watch to make sure all of our captions go through, and then I will end the webinar. [chuckles] 

[laughter] 

So thanks so much, Natalie.

>> NATALIE: Thank you.

>> CHRIS: Thanks for listening to Accessibility Craft. If you enjoyed this episode, please subscribe in your podcast app to get notified when future episodes release. You can find Accessibility Craft on Apple podcasts, Google podcasts, Spotify, and more. And if building accessibility awareness is important to you, please consider rating Accessibility Craft five stars on Apple podcasts. Accessibility Craft is produced by Equalize Digital and hosted by Amber Hinds, Chris Hinds, and Steve Jones. Steve Jones composed our theme music. Learn how we helped make 1000s of WordPress websites more accessible at equalizedigital.com.