130: WordPress Page Builder Accessibility Update, Warped Wing Fly Light Lager

By /

In this episode, we share our reasons behind making the WordPress Page Builder Accessibility Report, which was originally released in 2024, an annually updated report. We also teased some preliminary results from the 100+ hours of analysis that have gone into the 2025 report, so far. There will be more than a few surprises. Tune in to learn more.

Listen

Watch

YouTube video

Links Mentioned

Transcript

Chris: Welcome to the Accessibility Craft Podcast, where we explore the art of creating accessible websites while trying out interesting craft beverages. This podcast is brought to you by the team at Equalize Digital, a WordPress accessibility company, and the proud creators of the Accessibility Checker plugin, and now onto the show.

Amber: Hey everybody, it’s Amber and I’m here today with Chris.

Chris: Hey, everybody.

Amber: And Steve.

Steve: Hey everyone, how’s it going?

Amber: And we are super excited to be on episode number 130. If you are tuning in and you wanna access show notes or a full transcript for this episode, you can find them at https://AccessibilityCraft.com/130.

Today’s Beverage

Amber: We always start every episode with a beverage, of course. Chris, what are we drinking today?

Chris: Well, I am super excited about this because Steve was kind enough to run around his hometown in Dayton, Ohio and find us an Ohio beer and ship it out. So, I don’t know. Steve, do you want to introduce the beverage today since you went out and grabbed this and put it in the box and got it all the way to Texas?

Amber: Very legally we might dad.

Chris: Yes.

Amber: For everyone listening here…

Steve: Maybe.

Amber: We didn’t mail beer across state lines. Not at all.

Steve: I mean, you know, it’s kinda what mama doesn’t know won’t hurt her, right? What UPS doesn’t know…

Chris: That is a true victimless crime in this case because it’s just for your entertainment. We’re not, that’s not doing anything else.

Steve: They’re podcast promotional materials. So, I’ll tell you a little bit about why I chose this one. They have a lot of seltzers and beers and sodas and stuff. But the reason I chose this one is because you guys make me drink stouts all the time, like nasty beers all the time.

And I’m not much of a beer drinker. And, if I have to drink a beer I want a light beer. So I chose a light beer, and this is a very light beer. It only has 4% alcohol and 95 calories, and it’s, you know, not many carbs, right? Pretty low. Yeah.

Amber: It’s a diet beer.

Steve: 3.5 carbs. If you’re on a diet, you know this is the beer for you.

Amber: This is the summer I’m fitting in my bikini beer. Is that what this is?

Steve: Yeah. Yeah. So when I go to put on my bikini after this, I look good. So, yeah. So this is from a brewery called Warped Wing Brewing Company. They have a location not far from where I am. It’s actually north Dayton. And I think there’s an interesting history.

I think it kind of springed out of a, it was a side business of a a local design agency that it sprung out of, and so you can actually get ’em in the stores, some of their drinks, like in the stores and some of their pops and stuff.

But I went up to the actual location and grabbed this and it, and I chose it a little bit too, ’cause you know, it’s very American looking with the red, white, and blue. And it says Dayton sixth at the bottom, which is a sports organization that the Dayton Flyers, our local basketball team are part of. And I guess they, you know, they contribute to this organization that helps promote and fund local sports for college and maybe high school as well.

I think it’s college, but so I thought it was a good cause and it kind of represented Dayton. It says Dayton on it, and it’s light, so I think it’ll be good.

Chris: Yeah, it’s even got the tie in, as you told me to the Dayton Flyers basketball team, right in the name the Fly Light Lager. It’s kind of a cool all over overall concept.

I’m excited to try it.

Amber: It says share a pint, make a friend.

Steve: There you go.

Chris: I’m keeping my ice cold in true light beer fashion.

Amber: You gotta put it in a koozie. I keep forgetting to get my pint glass so I can pour them in. I’m always like, busy working. And then Chris just hands me a can or a bottle of beer right before we record and I just have to get on and run a stream.

Maybe I can find a koozie in my desk.

Steve: Yeah.

Amber: All right. How the smell, Chris?

Chris: It’s got that classic light beer, like kind of yeasty, a little sour smell. Yeah. It’s smells good, smells appetizing.

Amber: It is, it’s funny ’cause I like hardly ever drink light beer. I just think of light beer as like the college keg party beer. I don’t know why.

Steve: Well, I mean, it’s got the university tie in here.

Amber: You think they they drink this at the frat parties in Dayton, Ohio?

Steve: At the, yeah, at the University of Dayton frat parties. I don’t know that maybe they go with something a little stronger than four. What is it? 4%?

Amber: Yeah. You know, you know, it’s interesting down here in Texas where we have some restrictions because we’re a little more conservative. In like the state parks and stuff, you’re not allowed to have beer over a certain alcohol threshold. Like they have a percent ABV that is allowed and then you, if you go over that, even if it’s a beer, you can’t have it. So this I think would be State of Texas approved, safe to drink while you’re hanging out at the beach. Beach, in quotes by the lake.

Steve: Yeah. I didn’t even know you were allowed to have alcohol at the state parks, but I do see people drinking at we have, we don’t have beaches. We got lake beaches, right? So, it’s very light. It’s very easy to drink. It definitely tastes like, you know, when you go into the fridge when you’re a teenager and you steal your grandpa’s Old Milwaukee Best Light. You know, like it has that kind of flavor.

Amber: Yeah, that’s exactly what I was thinking. It’s this is what I imagine Duff beer from the Simpsons tastes like.

Chris: So it’s funny you mentioned Duff Beer from The Simpsons because we were just at Universal in Orlando and they had a big statue of Duff Man, and I have a picture of me posing next to him that I can post in our internal chat. But…

Amber: Why? I was just like, why is my phone not here? I would try and hold it up. I think when you edit this podcast episode, you should put the picture just as a fun little easter egg right here.

Chris: Maybe I can try to actually figure out how to do that this time. ‘Cause the last time we promised me putting in a cut, it didn’t go so well and I wasted two hours trying to figure out how to do it in Descript. And I gave up.

Amber: I can show you how to do it in Descript.

Steve: There you go.

Chris: I’m not saying this in a negative way. This is a hard beer to think deeply about and really analyze because it’s so simple. But I’m gonna try to describe it for people to, and not just a, well, it’s a light beer kind of way.

That is kind of what it is, but on the nose and on the flavor, because there’s like very little hop, very little malt, and the carbonation, I would say is fairly aggressive. Like it’s pretty darn bubbly in my opinion, compared to some other beers we’ve had. It’s just kind of got this white bread vibe, like on the nose and on the palate. Like it’s just like plain simple. Like yeasty, like you would like. Yeah.

Yeasty flavor, you would detect in like a loaf of just plain, nice country white bread that you would…

Steve: Midwestern white.

Chris: Yeah, and that’s, you know what, some days that’s all you want. You don’t want to think about your drink. You just wanna relax and chill out and have something that’s easy. Especially when you do all that big brain thinking like Steve, and we keep throwing these complex beers at him.

It’s you know, just gimme something easy for once.

Steve: Yeah. Yeah.

Amber: Are you saying that the appropriate food to pair with this is hot dish?

Steve: Hot dogs.

I’m thinking Midwestern…

Chris: Hot dog hot dish.

Amber: Oh. You know, those exist.

Chris: That probably does.

Amber: For everyone in Europe and outside of the Midwest. Hot dish is basically like casserole only, I don’t know, it’s like a, it’s like a glass pan that you’ve put random, weird things in all together with some sort of gravy or sauce and baked in the oven.

You know, your grandma would’ve put corn flakes on the top of it. At least in Iowa that’s what we did, because we love corn.

But yeah, that’s the food pairing that we would put with this?

Steve: Yeah. Well, I mean, you know, I think that Chris, you described Dayton, Ohio in a beer, and it’s very plain and white bread, and Midwestern, and…

Amber: I almost spit into my microphone.

Steve: But I will say this, you know, I’m not a, I’m not much of a drinker at all. And if I would have one like on a Saturday night or something like I don’t need much alcohol level in these to actually feel it. So four percent’s enough to feel it.

And it’s, you know, it being light, it’s pretty watery. So it actually feels like on a really hot day if you sat out on your back porch, you get a little bit of a quenching from this, you know, rather than some of the beers that we’ve had before where you feel like you’re chewing on ’em, you know?

Amber: So are you a thumbs up on this beer, Steve?

Steve: I mean, I’m a, probably, I’m a, probably thumbs in the middle on this beer. But you know, that’s very Dayton, Ohio, you know, to be thumbs in the middle, you know, just average…

Chris: I would say to put it in Gen Alpha terms, I don’t think that this beer is Ohio.

Steve: Yeah.

Chris: You know, as in isn’t how they use it as like a derogatory term, which I think is so ridiculous because we’ve been through Ohio. It’s beautiful and it’s nice, but yeah. Yeah. I don’t know where they’re coming from with that.

Amber: But we don’t live there.

Chris: I would certainly come back to this again, I think and try it again. I actually, what I’m curious about, would be to get at some point in the not too distant future, so memories are still fresh, like a different light beer, and then we have a contrast. It’s okay, is there a lot of difference or are all light beer is pretty much one note the same no matter who makes it.

And that might be an interesting experiment because light beer is not something that I have a ton of experience with. As we’ve established on this podcast. We tend to go for the heavier, weirder stuff.

Steve: Yeah.

Chris: This was cool. This was a deviation from the norm, so thank you, Steve.

Amber: I am also kind of a thumbs in the middle, ’cause to me it’s just kind of boring. So I don’t know if I would buy it again, but it’s not bad. We’ve had way worse things. It’s just eh. Okay.

Chris: We’ve had way worse things that we’re trying way harder than they probably tried to make this.

Steve: Yeah.

Chris: Which is kind of funny if you think about it.

WordPress Accessibility Page Builder Comparison: The What and the Why

Amber: So I thought it would be fun to talk about this huge project that I and our accessibility specialist Maria have been working on for the past month or so, maybe even a little bit more. Which is our WordPress Page Builder Accessibility Comparison Report which is getting updated for 2025.

Last time we talked about this, we had a spicy beverage and I definitely recommend that everyone should go and take a look at that episode, which we’re gonna link in the show notes for this episode. If you wanna hear more about what the results were for 2024 but we can talk a little bit about those too.

Steve: So it was spicy last time, but that was after the fact. We were talking about the results from the page builder report, right?

Amber: Yes, that is true. So that one actually we shared thoughts and data.

Steve: But this is before the page builder report. So you know, for those that didn’t listen last time and are listening this time, what exactly is the Page Builder Accessibility Comparison?

Amber: It is a report that goes through the accessibility of a bunch of popular page builders. Looking at how different components compare against WCAG criteria, so things like if they have a tabs block or tabs widget, which they use different words, but those, so we look at all of the tabs, components for all the different builders and give them passes, fails, or concerns on, everything that you would need to do for a tabs, for example. So we look at a bunch of different components and I have them all in a spreadsheet, and then in the end, they end up getting a score that is a percentage score of how much they passed on applicable things.

So they won’t lose points if they don’t have a tabs component, for example. We’re only gonna count how much they passed on the things they actually built and then from that we’re able to rank them from best to worst from an accessibility perspective.

Chris: And this is kind of a two part question. Do you remember a year ago, like what led to you deciding to do this originally? And then maybe you can follow that up with why did we decide to keep doing it and make this a regular thing?

Amber: Yeah, so the original reason why I started doing it is because at the Meetup everyone always asks us this, and in our Facebook group, our WordPress Accessibility Facebook group, which anyone can join, a lot of people will say, what’s the best page builder for accessibility, or what’s the best starter?

And our history is that we were building lots of custom stuff. Steve always did like a custom theme. We didn’t really use builders or libraries. I mean, we hadn’t, for years and years, we only really touched them if a client came to us that had it and we needed to make some modifications. So I didn’t really know.

And I originally started ’cause I’m like, I wanna be able to answer this question. When I did it last year, I didn’t necessarily know if it would be an annual thing and I didn’t do it in a way that made it easy to make it an annual update. I have corrected for that this year. Yeah, the goal now is that this can be an annual report.

And part of why it’s annual is because things change. WordPress plugins update all the time. And so I wanted to be useful information for people and also it’s not super fair if someone scored really low for them to say low on our chart, because there were some builders that did not do well at all.

But then they immediately turned around and started making fixes and so they might now be outranking someone who was higher than them last time. So I realized that it really does have to be updated annually. I just, logistically, because of the amount of time, I don’t think we could do it more than annually, but it will definitely be an annual report that we put out every year.

Reminiscing About the 2024 WordPress Page Builder Accessibility Report

Steve: So talking about some of the results from last year, what were some of the key findings that you noticed last year?

Amber: Yeah, so, in the, we’ll link this in the show notes also. You can actually go on our website and see the full report and get access to that spreadsheet of data. It’s going to be there, it’s gonna live forever if you wanna be able to compare and see how things improve.

The short summary was last year we did ten page builders. Number one was Kadence WP. Elementor was number two. And then down at the very bottom was Beaver Builder and number nine and Divi at number ten. I think it, it was interesting to see like some of the things were almost everyone failed, like sliders not having pause buttons when they autoplay.

And it is just interesting to me that nobody even thought to add that as an option, except for, I think Beaver Builder was the only one who had that. And I think there is definitely a very wide variety in how different builders handle things that maybe surprised me to a degree. But there were also a few when we looked at carousels, where we could tell from looking at the code, they’re all using the like, five of them were using the same library to build their carousels.

And so as a result, they all had the exact same problems. So if that library were to fix problems, then all of those builders that use that library would have been more accessible. I don’t know. Did anything jump out to you guys from last year’s report?

Steve: I mean, definitely the carousels. I mean, there were like focus shifts that seemed to be problems in most of them last year. Like focus shift to slide when using navigation buttons. I’m guessing that’s yeah, that’s in regards to carousels as well, and sliders, that, that seemed to be the biggest category where things were kind of, brought, you know, a lot of issues across the board was in relation to the carousel components.

Amber: I will say there were a few builders that it shocked me like I expected with accordions and tabs that we might see a lot of implementation where there were things like links instead of buttons. So they still seemed kind of functional with the keyboard, but once you listened to it with screen reader, it’s really apparent that it’s missing important information.

But there were a few, like Bricks was one of ’em, where their native components did not function at all. Divi is another one because they just used divs and there was no tab focus set on it. No tab index, no ARIA. You couldn’t even tap to it with a keyboard. And so I think that was a thing that surprised me in 2024 that there are major builders that are used by, in some cases, hundreds of thousands or millions of websites that literally have navigation menus or accordion blocks or something like that, that can’t even be interacted with a keyboard.

I expected there that everything would have some sort of basic keyboard functionality, but might just not be great for screen readers. So that surprised me last year.

Steve: Yeah. Yeah.

Do we think the rise of AI code helpers will mean better scores in 2025?

Amber: But I don’t know, do you guys think this year that with the rise of AI, ’cause that is a huge difference between when we did this test and I, most of the auditing took place between June and July of 2024 and this year we’ve been doing it between June and August of 2025. And now we have AI code helpers.

Do you think that we’ll see better results because of that?

Steve: No. That’s a simple answer. ’cause the AI is, I mean, I don’t want to go too deep into a rabbit hole on LLMs, but they’re not as intelligent as artificial intelligence label would imply they are, it still requires a human. Now, if they took this report and fed it into the LLM and with the context of their code base, then sure, I’m sure they, it can generate all kinds of issues and help you handle those.

But what I’ve noticed, I mean, and we’ve talked about this kind of ad nauseum, is that, you know, accessibility really on the surface, some of the things that we’re talking about are not difficult to fix at all. It’s just you have to have the mindset and you have to give it the attention and the priority.

So I don’t, and I don’t know that is gonna set priorities for you. Now, it can help you implement those priorities, right?

Chris: I kind of wonder too, and this is something that we may have alluded to previously, is the issue of the data that these models are trained on, which when you have a 95% inaccessible web, and that is at least accounting for probably the majority of the training data, maybe even for the code.

I’m not sure if that’s a lot of synthesized training data or how that training data’s being generated or aggregated, but, if you have inaccessible training data and then telling the model to generate code for you the likelihood is it’s going to generate at least partially inaccessible code, potentially.

Steve: Yeah. I mean, it depends how you prompt it, right? I mean, from my experience, it does a fairly good job at doing it. But no, I can’t speak to whether or not it’s that what it’s trained on is correct or that my prompting is correct, right? And I kind of, a lot of times when I we’re remediating things, we kind of know what we’re looking for.

We even, you know, a lot of times we even have, you know, WCAG criteria that we can actually plop in there and then it’ll use the WCAG documentation to kind of come up with a solution. So I think that at that point it normally is pretty good once you prompt it, but I just think that development teams have to actually still prioritize this stuff.

Chris: Yeah. You have to define, have, give it a clearly defined prompt that prioritizes that to begin with. Yeah.

Amber: Yeah.

Chris: I know that this year’s report includes more builders. I think we already said that. And maybe we can reveal what additional builders are coming right after this short commercial break?

Brought to you by Accessibility Checker

Amber: Sure, let’s do that.

Steve: This episode of Accessibility Craft is sponsored by Equalize Digital Accessibility Checker, the WordPress plugin that helps you find accessibility problems before you hit publish. Thousands of businesses, nonprofits, universities, and government agencies around the world trust Accessibility Checker to help their teams find, fix, and prevent accessibility problems on an ongoing basis.

New to accessibility? Equalize Digital Accessibility Checker is here to teach you every step of the way, whether you’re a content creator or a developer, our detailed documentation guides you through fixing accessibility issues.

Never lose track of accessibility again with real time scans each time you save, powerful reports inside the WordPress dashboard, and a front end view to help you track down hard to find issues. Scan unlimited posts and pages with Accessibility Checker Free. Upgrade to Accessibility Checker Pro to scan your website in bulk, whether it has 10 pages or 10,000.

Download Accessibility Checker today at EqualizeDigital.com/Accessibility-Checker. Use coupon code AccessibilityCraft to save 10% on any plan.

Chris: So, Amber…

Amber: We’re back!

2025 Page Builder Accessibility Report Preview

Chris: You had thousands, maybe millions of people beating down your door and flooding your inbox saying review my page builder. What small percentage made the cut?

Amber: I don’t, I, Chris winked for everyone who’s listening to this episode. So we did actually have a few builders that contacted us, which I love and asked to be included. So shout out to WP Bakery. They came up to me at WordCamp Europe and they’re like, I’ve heard that you’re going to do this thing again. Can I give you a free version of our plugin?

I love that. I think that is really cool that people are asking for accessibility feedback because what that means is they’re probably going to prioritize the fixes. There have definitely been some builders that I’ve reached out to and it’s been crickets.

Which makes me a little more nervous. But yes, I did decide this year that we needed to expand beyond the original ten. The original ten were largely selected either because we already had a customer that used that on their website, so I could get a copy of the plugin, or I looked for ones that had free options.

And actually it’s interesting because if you go and search page builder on WordPress.Org, that was how I learned about Site Origin, page Builder, which has been around forever and is actually installed on huge numbers of websites. They’re all probably really old websites because it’s using like widgets to build everything. That’s how old it is, right?

Chris: Oh, I remember widgets.

Amber: Yeah. But it’s on like hundreds of thousands of websites. So, so that was my original ten. I asked in our Facebook group what people wanted me to test this year. And we are up to twenty. I don’t quite have them all set up yet, so we’ll see if we end that twenty timing wise, because we only have about a week and a half left to finish all of the testing and comparison.

But that’s my goal is to have twenty different builders. It will include Avada, Beaver Builder, Break Dance, Bricks, Brizy, Divi, Elementor, Generate Press, Green Shift, GREYD, Kadence, Live Composer, Oxygen, Seed Prod, Site Origin Page Builder, Spectra, Themeify, Thrive Architect, visual Composer, and WP Bakery, if I get them all done.

So, so that is the tentative list right now. If you’re listening, you can find out in one week when the, we haven’t said this yet. I can’t believe we’ve gotten this far in the episode and not said that the report’s gonna be available in one week from the day this podcast comes out.

Steve: So interestingly enough, there’s some in here I haven’t never even heard of. Like…

Amber: Me too. I’ll say that’s one of the things I love about this project, like doing this, there were a lot of tools that people said, please test this. And I was like, I’ve never even heard of that. And then I get to go try ’em out and play with them, you know, setting them up.

Sometimes I’m frustrated doing that, but sometimes I’m like, oh, this is actually kind of a neat tool. Like it’s interesting. I’ve had that same thought. Footnote, but like we’ve been auditing all of the WooCommerce extensions and I’ve been like, wow, there’s some really cool WooCommerce extensions that I had no idea existed.

So it’s kind of fun to just go out and get out of your bubble of your WordPress plugins and try other ones and see what’s out there. Yeah, but did you notice what one, what one I did not say?

Steve: Yeah, Gutenberg the block editor? So why is that not included? We didn’t do it last year. We didn’t do it this year. What’s the deal?

Amber: So everyone asked me that last year. Why isn’t Gutenberg on this? I did actually go look at the Gutenberg plugin, which I don’t have time, so I don’t really follow like what’s in the Gutenberg plugin that might be coming to core later. I did install it to try and see like what blocks exist there, but the reason why I haven’t included it is because Core and Gutenberg don’t have a bunch of the components I am looking at.

No carousels, no accordions, no tabs, like no list icon block, testimonials block doesn’t exist that allows you to put in ratings or anything like that. There’s just so many components that don’t exist in Gutenberg that I don’t, I feel when I was originally deciding what would I test, I was thinking about what are the things that, when we were getting custom designs from clients that were almost always included in like a typical business website. And that is those things, tabs, accordions, as much as we hate them carousels, right?

Like those are almost always on all of those business websites. And so, I landed on, there’s no point in including Gutenberg because it doesn’t have it. So people are going to go out and find a different builder in order to build those things, or they’re gonna make ’em custom. But so I just, I still think in 2025, so one year later, I’ll repeat what I said last time in our super spicy episode is that you cannot build a professional business website with core WordPress alone.

Steve: Yeah. And so a little follow up on that and so actually Gutenberg would probably look quite well in this report because they’d have a lot of N/A’s, right? Like a not applicables, but…

Amber: I mean, I do think the blocks that are in it are accessible because the accessibility team tests them. And I think if I did a core theme, it would probably, it would pass the accessibility ready review, which is something that I added this year more details on.

Steve: But you have to go, you have to go out and you have to get third party blocks to really be able to achieve what a, I mean, not just an enterprise website, but like any normal website would want, maybe a slider or a tabs or an accordion and, you know, things like that. And then, and what that does is that introduces a third party accessibility consideration, right?

Amber: Yeah, I mean, honestly, it’s interesting. Not that we’re trying to make this one as spicy as it was last year but I, I do really think as much as we’ve been talking about with WordPress core, you know, oh, let’s make collaborative editing possible, or whatever it is that’s in 6.9, right? Or wait, whatever version of WordPress we’re working on right now. Like I still go back to, you literally can’t build a website other than a blog with core WordPress blocks only. You either have to build them custom or get a third party.

So I don’t know why we’re doing all this fancy collaborative editing when we don’t even have the blocks that you need to build a nice looking website.

Steve: Yeah. Yeah. We’re getting spicy now. We’re getting real spicy.

Chris: Yeah. We might need a second beer to cool down. I know.

Steve: Good thing I have this watery beer to…

Amber: Get me to chill out a little bit. Just a little bit.

Steve: Just a little bit.

Chris: So, Amber you alluded to this very slightly when we were talking about Gutenberg and some different requirements that you’re gonna surface more in the report. So how are our reports going to be different this year or what’s changing?

Amber: Yeah, so I did add one component, form blocks. Last year, I intentionally did not look at any form blocks because I still feel pretty strongly from a personal standpoint that you should just use a form plugin. There’s a lot of form plugins that do amazing things and I feel any form from a builder is not, it’s gonna be so limited or lackluster that I didn’t wanna do it.

But this year as we were discussing internally, and I was reflecting on the reality of how easy is it to make an accessible website for an average user, not a developer with a, the builder, the reality is I can’t ignore the form block. Because all the starter templates are gonna use it, it can’t be ignored.

So I, we are testing the form blocks if they exist in any of these builders. That is probably the biggest from a component standpoint. The other big change is just in how the report is going to look. So last year was a single page the spreadsheet which you will be able to get access to if you register for our webinar or visit our website after Monday, June 18th is going to actually have three tags.

Chris: Sorry, Monday, August 18th, right?

Amber: Oh yeah. Did I say June? August 18th. But yeah, so it’s going to have three tabs and there’s gonna be a summary at the front, the components are all on one, and then I had to pull the accessibility ready requirements out into a separate tab.

We should probably follow up discussion on the podcast about how the accessibility ready requirements in WordPress are changing. But it’s a lot more detailed than what I had last year because I put a whole bunch of effort for the WordPress project and the accessibility team in modifying these.

And so, it went from being like, I wanna call it like a 18 or 20 point checklist to, I don’t even know how many rows are on this spreadsheet. Now, let’s see there’s 205 rows on the spreadsheet for accessibility ready. So it is a much more detailed audit that people are going to get access to. And a lot more thorough than what it was last year.

The other big thing, I mentioned this earlier that last year, I wasn’t planning ahead for being able to retest. This year, the way I have set it up is different than last year. And maybe we could talk about that testing process. But basically you’re also going to get access to my testing sites if you wanna look at them yourself.

And I started this out because last year, the Bricks community in particular, I got so many DMs and like messages and people being like, this is bunk, it’s totally a lie, you just don’t know what you’re doing, you can’t use automated tests only. And I’m telling you, we don’t use automated tests only. Like they did not believe me at how bad Bricks was.

So this year there will be public web pages where you can literally go look at what our team tested, the exact component in the exact configuration, as proof, that we’re not making stuff up.

Steve: Yeah, it’s interesting how passionate people can get around page builders sometimes. And but yeah, I mean, like you said, like that a little bit of insights into the testing process and how it works and how much time it takes and like you mentioned too, like we’re using our internal resources to, to do this. Can you speak to that a little bit?

Amber: Yes. So the biggest thing that’s different this year, luckily, is that I’m not doing it alone. So our accessibility specialist Maria gets a huge shout out for doing a ton of the tests. The way it’s working now, last year I just did it on my local, like I had a local WordPress install and I kind of overwrote it each time and I would install demo content from whatever.

Because I was trying to get like the fastest results. So if they had to install a demo, but then the content was always different. What I decided to do this year was I actually created a WordPress multi-site, and every subsite is a builder. They all have the exact same content on them.

I duplicated the subsites to create them all. So they have the exact same blog posts, with the exact same images, the exact same text and links and all of that kind of stuff. So that’s what people are gonna be able to access and you’re gonna be able to see with the exact same content put into a query loop block, for example. Like how does it compare?

So there’s a lot of work just in setting all of that up. I would say for a builder I’m familiar with, I can get it set up like inserting the components and making sure everything is set up in about an hour. For one that I’m not familiar with, it could take me two or more hours depending on how hard it is and if I have to Google things to try and figure out why I can’t find a setting that I expect to find or something like that.

And then Maria has been doing an initial pass through. We do automated tests with Accessibility Checker. We will also be comparing, nice thing about our, it being in a multi-site was I could use our Accessibility Checker Multi-site Add-on. So in the first tab of the spreadsheet, you’re gonna be able to see all of the Accessibility Checker reports.

And I’m hoping, I think we have a filter in this plugin that will allow people to see Accessibility Checker, even if they’re logged out. So, so we’re gonna enable that on this test site so you can not be logged into it, but still see the Accessibility Checker reports on each of these pages. So we’re setting all of that up.

And then in addition to the automated testing, we are of course doing manual testing because as we talk about all the time, you can’t find everything with automated. So Maria’s using NVDA, I’m double checking things if she’s not sure about them with VoiceOver. So we’re getting coverage on both Windows and Mac.

And we’re doing manual keyboard testing, we’re doing testing on zoom. We’re even testing like going in Firefox browser and toggling the zoom text only setting and looking at it. So we are being incredibly thorough because we really wanna provide an honest view of how all of these page builders stand up.

And where we are for time right now, you, neither of you guys, I don’t think know this, so this will be fun to hear your reaction. We are, I would say at the time of recording this episode, about three quarters of the way done with our testing and set up, but we still have to do setup and testing of the others, and we have already put in over 110 hours into this.

Steve: I think we’re gonna have…

Amber: A surprise to you guys not doing billable work.

Steve: Yeah, I think we’re gonna have to have a, an emergency executive meeting after this podcast recording.

Amber: But I, you know, the thing is, it is a huge amount of time to put together a report like this. But I think it’s so important for the WordPress community and to me, like it is worth us doing this time and putting it in. And because it really does make a difference. And like I said before, there were builders that last year, they were like, oh crap, I need to fix my stuff.

And so to me, I’m like, okay, that’s good for the community. I mean, in theory, everyone listening to this will also decide to go join our webinar and maybe join our email list if you value the effort and the time that we have put into this. Maybe give Accessibility Checker a try. But yeah, it’s, it is a huge time sink to set all these up and then audit them thoroughly.

Chris: Well, and these weren’t even true comprehensive audits of each builder because there’s components and things we didn’t test that in the scope of an audit for a forward WordPress product, we would’ve evaluated much more exhaustively even than this. Because…

Amber: Oh, for sure.

Chris: 20 builders, 120 hours divided by 20 builders is maybe a little over five hours per builder. So that’s definitely a solid surface level look at some of the major stuff, but we would be going, we would be spending that equivalent amount of time or more for each individual builder if we were doing a full audit. So I wanna tag onto what Amber said because it’s an astronomical amount of work and data that we’re putting out there for free.

So if you do value this work please share it, within your communities. Please help get more eyeballs on it. If you’re, if you think the builder you’re using, might not be aware of it, send it to them. And if the builders looking at this report value the work that we’re doing and want deeper accessibility work done, reach out to me or to Amber, or even to Steve, and we can have a conversation. And try to help you more deeply.

So I mean, it’s an astronomical amount of work, but I think hopefully for us it will pay dividends in some fashion and certainly uplift…

Steve: Hopefully. Well, speaking of an astronomical amount of work, a hundred plus hours already in it. Can you tease anything? What’s, can you give us anything yet?

Amber: Yes. So I will say I’m looking at my report to see what I can tease. I’m trying to look at, I’m trying to look at Bricks because they were one that like so many people got so upset about how low they were, and I know that they reached out and they were like, Hey, we are going to work on stuff. So one of the things that I could say, like last year, their accordions failed everything. They weren’t keyboard focusable, and now they use buttons instead of divs. So that is amazing. I love that. So you can expect to see their score go up in this report.

I think the Kadence score is going up. They were the top, but they weren’t 100%, right? And I know they had said to me that they had a goal, like even last year, Ben messaged me afterwards and he’s I wanna be 100% on your report. So you’ll have to turn in to find out if they made 100%.

But they, you know, there are definitely a bunch of builders that put some effort in. Which is exciting and I love that. And it makes me feel like what we did, you know, doing this makes a difference not just for us as a company, but for the whole WordPress ecosystem.

I could also tease, this was a pain point that I flagged last year, which was that every builder did not have the ability to turn on a pause button if you autoplay a carousel. And I, except for Beaver Builder. And I literally said this when we were talking on this podcast when we did the meetup to present the report. And I was like, I don’t understand why Beaver Builder is the only team that has decided this should be a setting.

This year, pretty much all of them also still don’t have that, so they’re not listening enough. It’s just like pause button on that autoplay. I don’t understand why that’s so hard, but maybe that is hard. Steve, do you have thoughts on that? Why the builders are ignoring creating pause buttons?

Steve: No, it’s not hard, but it, what it probably is it’s probably outside the scope of the slider JavaScript packets that they’re using. So it would be a custom thing that they would have to lay on top of on top of whatever they’re using. So..

Amber: So it’d slow ’em down.

Steve: Well, you know, the hard thing with page builders and making changes from an accessibility standpoint , why it’s so difficult is for backwards compatibility, right?

Like you, you think if somebody’s using a div, you just have ’em flip it to a button and you’re good to go. But really hundreds of thousands of websites are using this. They could be targeting that diviv directly in some weird, you know, parent child relationship way, right? And it, and you modify the selector and it messes it up.

So in that case, ideally you would probably just put a role of button on that div and then create some JavaScript to kinda make it act like a button. In most cases, if they did add that pause button, it would probably have to be, if it’s a widget that’s already in their builder, they’d probably have to be opt in, not default.

Or maybe it’s only default if it’s a widget or block that’s added to the page brand new. Right, like not existing ones. So they have to think about that stuff a lot and I can’t imagine how, difficult it is to release updates for, you know, plugins like Elementor, right? ‘Cause they have breaking changes and then the whole internet blows up for a day, right?

So, yeah, it’s hard. I mean, it does kind of lead me to wonder, hey, maybe we need to build that into the Accessibility Checker fixes to add a pause button to sliders that use certain Java script sliding classes. But…

Chris: You heard it here first.

Steve: You heard it here first.

Chris: That’s pretty good.

Amber: All right, we’re gonna go create a GitHub issue after we finish recording this. Hold you to it.

Steve: There you go.

Chris: I gotta say what is most interesting to me about this and what has me most excited to see the final results is like what the two bookends are going to be on this ranking. So who’s gonna be highest, who’s gonna be lowest? And right now looking at the distribution of the percentages as testing is still ongoing.

Amber: You can’t say.

Chris: I’m not 100% sure who, and I mean that honestly you’re saying you can’t say it’s already a done deal. I’m not 100% sure because the testing’s not done, the weighted scoring’s not done. And there are multiple builders at the top and the bottom that I saw scrolling through like some of the preliminary results we have right now whose percentages are very close.

There’s like at the low end, there’s like a 27% and there’s a 31%. And there’s like a 29%, and then at the top end there’s like an 81 and there’s something a little higher, something a little lower, like there there’s some people that are really neck and neck for coming in first and coming in dead last.

And I don’t know that it’s gonna be the same ones.

Amber: Yeah. So that, it’ll be really interesting. I’ll say I also still really find interesting how big of a gap there is between like the top and the bottom. There are some tools, companies that get it mostly right. And there are some that just about fail everything, and then there’s this gap in between.

Steve: So for my own curiosity there any that have gotten worse or is this actually rising, the tides of everybody, like kind of the this being kind of public and maybe creating a little bit of a competition, a little bit of a priority around accessibility, a conversation around it, at least.

You talked about Bricks, right? You know, they’re not happy, but at least we’re having a conversation. So what are your thoughts? Is anybody getting worse or is everybody just seem to be getting better or the same?

Amber: So I will say it’s gonna be a little hard to answer that because only because I added more tests in

Steve: Yeah.

Amber: So we can use Divi as an example. Last year, if you look at the report, Divi scored 16%. They were the bottom by far from everyone. On only my component tests, and I don’t have these combined yet, which is why it also makes it hard. Like I literally don’t know the answer because I have them on two different tabs , I haven’t built my summary sheet yet that it combines the scores.

But on the component test, they’re scoring like 24%. And on the accessibility ready test, they’re scoring like 50%, which makes it sound like maybe they’re gonna average up and they will have gotten better, not worse. But what’s a little weird about that is that we added in some extra things, like I didn’t look at forms.

Maybe their forms are actually pretty decent, and so that averages out the fact that they don’t have focus indicators on anything. You know, I, I don’t know, like mathematically…

Chris: I don’t know that’s bad either though. If their score, if anyone’s score is inflated, just because we’re testing more. If anything, that’s just a better picture of reality compared to the more limited scope testing we did last year.

So if they happen to go up, it’s just, it is what it is. But I think from this point forward, year to year tests might stay a little bit more consistent, at least for a time, given all the infrastructure we’ve built around having a consistent system to keep going back to.

Amber: Yeah, hopefully.

Chris: We need to do it one more time. We need to tell everyone how this data’s coming out, when they can get it, where they need to show up to get it before everybody else. All that good stuff.

Amber: Yep. So, the easy way to point everyone there is if you go to the show notes, which are AccessibilityCraft.com/130, you will be able to find a link in the show notes to register for WordPress Accessibility Meetup, which is on Monday, August 18th at 7:00 PM US Central time. And in that webinar, it’s a Zoom webinar, so you just have to register on Zoom.

I’m going to reveal all of this and share a link so that you can access the spreadsheets and the data. We will also, if it is after the fact when you’re listening to this, we will also be publishing this on a blog post on our website, and there will be a form that you can fill out to get access to the full data from the summary blog post, and that will be able to be found at EqualizeDigital.com/WordPress-Page-Builder-Accessibility.

So the current 2024 report will stay available. It’s gonna get slash 2024 appended to it, and that main URL is the one we’re gonna update every year. So if you go there, which is also linked in the show notes after August 18th, you will also be able to access the information and the date.

Chris: Amazing.

Steve: Awesome.

Chris: And thank you. And thank you, Maria for all your hard work in getting this together and getting it over the finish line because it’s not done yet.

Amber: Yep. Yeah, we have a lot of testing to do in this last week, so, it should be really fun, and I also cannot wait until the 18th to be able to reveal it and show who is the best and who is the worst at accessibility in WordPress.

Steve: Awesome.

Amber: All right, well, we’ll see y’all back here for another podcast episode in two weeks. Bye.

Steve: See you.

Chris: Thanks for listening to Accessibility Craft. If you enjoyed this episode, please subscribe in your podcast app to get notified when future episodes release. You can find Accessibility Craft on Apple Podcasts, Spotify, and more. And if building accessibility awareness is important to you, please consider rating Accessibility Craft five stars on Apple Podcasts.

Accessibility Craft is produced by Equalize Digital and hosted by Amber Hinds, Chris Hinds and Steve Jones. Steve Jones composed our theme music. Learn how we help make thousands of WordPress websites more accessible at EqualizeDigital.Com.