In this episode, we discuss best practices for surveys and collecting user feedback by reviewing a user survey about accessibility that was released by a UK nonprofit in January 2025. We discuss timing, content and types of data to collect, communication strategies, and things to do before a user feedback survey goes out (particularly if that survey is about the usability of your website).
Listen
Watch
Mentioned in this Episode
- High Noon Vodka Seltzer Mango
- Big Brother Watch User Feedback Survey
- WAVE report for Big Brother Watch survey page
- User-Feedback Requests: 5 Guidelines
Transcript
Chris: Welcome to the Accessibility Craft Podcast, where we explore the art of creating accessible websites while trying out interesting craft beverages. This podcast is brought to you by the team at Equalize Digital, a WordPress accessibility company, and the proud creators of the Accessibility Checker plugin.
And now, on to the show.
Amber: Hey everybody, it’s Amber, and I’m here today with Chris.
Chris: Hey everyone.
Amber: And Steve.
Steve: Hello everybody.
Amber: And this is episode 106. So, if you want to find show notes and a full transcript, you can go to AccessibilityCraft.com/106.
Of course, we start every episode with a beverage.
Today’s Beverage
Amber: What are we drinking today, Chris?
Chris: We are trying out High Noon Spirits Vodka Seltzer Mango flavor. Sometimes colloquially known as Nooners, apparently. Short for High Noon, but I, I’m hearing, I’m hearing that these are making waves as being a really good seltzer. And their claim is it’s, it’s vodka, so not miscellaneous grain alcohol, but vodka and real fruit juice as the two basic ingredients.
Amber: This already wins for me. No high fructose corn syrup, no artificial flavors. This is my all natural person loving the green washing on their can, but it’s in blue font, not green.
Steve: It just has cancer causing alcohol.
Amber: Oh yeah, I know. Why did that study have to come out?
Steve: I know.
Chris: For health reasons we might have to make an alcoholic beverage be every third conversation episode instead of every other.
Amber: We’re going to need to cut back. Although this is the only time I actually, so I think I drink about once a month at the cadence that we record these and it is during this podcast episode.
So I think I’m okay. Well, so this can is a skinny tall can. I don’t know if there’s a name for these kinds of cans, but they, I feel like they’re getting more trendy now.
Chris: I just call them skinny cans. I don’t know what else to call them because they’re not minis, right? The minis are like the super tiny ones they have in Europe. It’s like a skinny can.
Steve: Yeah.
Amber: Well, why do you think people are doing this now? Making these tall cans?
Steve: I actually think, I don’t, I don’t know all the details, but I’ve heard there’s some kind of patent either with the like traditional Coke sized can or with this one. There’s something, we’ll have to dive into it.
Amber: Oh really? There’s like politics on everything.
Steve: Oh yeah, yeah, yeah, yeah.
Amber: Everything has like, problems. Their label kinda looks like a sun, their logo, with all caps. High on one line and Noon on the other. And I, I thought the back was sort of interesting. I always like to see the interesting thing that product companies put on there, but it says, “Looks like you’ve made it to the back of the can. Not a ton happens here. Most of the good stuff happens up top. But it gives us a chance to compliment you on your choice of hard seltzer. You picked a good one. A seltzer crafted with real vodka and real juice that’s also a can shaped reminder to always live like the sun is out. And since you’ve got a high noon in hand, we’ll wrap this up so you can start enjoying.”
Chris: Alright, so we can try our, our mango nooners. Ours are, it says best served cold. Is yours chilled, Steve?
Steve: Yeah, mine’s chilled.
Chris: All right.
Amber: All right. Sometimes you show up and you’re like, I forgot to put it in the refrigerator.
Chris: Wasn’t that the pickle beer?
Steve: I think it was.
Amber: That was like the worst one. Oh, man.
Chris: Oh, man.
Amber: Oh, it smells good.
Chris: I’m like a foot away from the can and I instantly smelled mango the second I cracked it.
Amber: Yeah, you don’t even have to hold this close to your nose to be able to smell it. I like the mango. Do you like the mango smell, Steve?
Steve: Yeah, it’s good. Now, I will say that. I mean, I haven’t heard it called a Nooner.
I like Nooners, but I’ve always thought of them as something else. Like…
Amber: Okay, this is a family friendly podcast.
Steve: I don’t know, is it? But I will say, when I go to the beach, I see these things everywhere.
Chris: Do you?
Amber: Oh, like this brand even?
Steve: Yeah, yeah.
Amber: I’ve never heard of this brand before.
Steve: Oh yeah, they have this at most of the stores now. Yeah.
Amber: And all the Dayton, Ohio beaches.
Steve: Well, no, when I go to the beach. We do have beaches! We’ve got the Great Lakes up at the top.
Amber: Oh, yeah. That’s that’s a cold beach, especially today I bet you.
Steve: Think it’s an ice beach today, but…
Amber: So I will say, I looked this up and it said it’s a great Summer beverage. And I was like, okay, well, we’re going to drink it and pretend like it’s Summer.
Chris: Oh, it’s, it’s it’s really good. It’s one of the better seltzers I’ve had. Not a lot of that weird alcohol undertone. Like, yeah, I don’t get almost any alcohol at all.
Steve: Yeah. This is dangerous.
Chris: Yeah. Yeah, and it’s four and a half percent. So it’s like about the equivalent of a mid range beer in terms of percentage.
Amber: And it’s not too sweet. I like this. It’s almost got kind of a dry flavor to it, but it’s really good. I put this up there with that prickly pear spiked seltzer that we had when Ronnie was on. That episode, I thought that one was pretty good. And I would say this one is, is good as well.
Chris: Well, I kept hearing about these. This is my first High Noon and I’m impressed. And I’m excited to try more flavors of this now. It’s pretty good.
Amber: Yeah. So, thumbs up, one, two?
Chris: I’m a, I’m a, I’m a single thumbs up. Cause, just because I’m, I’m usually, I don’t gravitate towards seltzers all the time.
Amber: Oh, I kind of like them sometimes. I would definitely, I would buy this again. Like if I was at the store when I was…
Steve: It’s not too bubbly though. I’ve had some that are so bubbly, they…
Amber: Oh, they’re like…
Steve: Yeah, it’s just way too much. It’s very good. I could see, you know, spring breakers getting really drunk, real quick on these things.
Amber: So the next time we go to the beach, this is what we should take.
Steve: It, like, like Chris said, I can’t taste any alcohol.
Chris: Honestly though, like, I’m going to a board game night with friends tomorrow night. I might pick up a four pack of these and bring them down with me because these are pretty good.
Steve: Yeah, yeah.
Amber: That would be a switch. You guys usually drink whiskey when you do board game nights.
Chris: Yeah, yeah.
Amber: Are the guys going to be like, why are you showing up with chick beverages?
Chris: And I’ll be like, just try one. Trust me.
Steve: Yeah. Like, I’ll drink this, you drink the whiskey, enjoy your drive home.
Amber: He marks himself on our family calendar and in a way that it’s such that I shouldn’t expect that he might not just sleep on his friend Carl’s couch.
Steve: There you go. That’s a good night.
Chris: It has happened once or twice, if I have more than, more than like one and a half drinks, I’ll just be like, I think I’m going to stay here.
The UK Nonprofit User Survey That Prompted Today’s Discussion
Amber: All right. So, I was doing some exploring for this podcast and I usually will sometimes go look and see if anything new has popped up with certain keywords. And I came across a user accessibility feedback survey, that a non profit in the UK called Big Brother Watch, which I’ve never heard of before, it’s all about helping you keep track of Big Brother watching you, put out on their website.
And I thought it would be sort of interesting to talk about. So the topic for today’s episode is: When Should You Ask for User Feedback?
I want to talk a little bit about this survey and whether we think that they’re asking for user feedback on accessibility in the right way, in the right timing, all that kind of stuff.
Chris: Cool.
Steve: Awesome.
Chris: So, I, I was taking a look at this. Would it be good for us to maybe start by just describing the survey and reading off the questions?
Amber: Yeah, I think that’s good. Do you, do you want to do that?
Chris: I volunteer as tribute, if we want to …
Alright, so the, the, I’m not, I’m not actually sure, is it an H1 on the page? Okay, well they’ve got that going for them. The H1 on the page is: Help Us Improve Our Website.
And they say Big Brother Watch wants to hear from you to give us or to give users a meaningful, easy website experience. If you require accessibility accommodations, your response to this survey will be invaluable.
Web accessibility features aid users who experience poor vision, mobility challenges, cognitive loss, and or other disabilities. We look forward to your response to this survey.
High level, I think that’s an okay intro. They go on to ask, these aren’t numbered, so I’m going to have to count here. It’s like six open ended questions with text fields for responses.
So the first is, how easy or difficult is it to read text on our website? Then, how easy or difficult is it to click on buttons on our website? How easy or difficult is it to access information about our campaigns on our website? Do you use assistive technologies to access information on our website?
And then they have two more What kind of accessibility or disability accommodations would you like to see on the Big Brother Watch website? And finally, do you have any other thoughts or comments about our website?
And then just a submit button.
Amber: And what is worth noting about this is all of them except for that final question are required. I mean, I haven’t submitted the form, but they have a little asterisks. So my assumption is they’re required.
Steve: Yeah, they are. I submitted it. They are. Yeah.
Amber: Okay. They are actually required.
Steve: Yep.
Amber: Programmatically, yeah. So I think, first of all, we probably want to applaud everyone for asking for accessibility feedback, and I think this is great.
And I don’t, I’m not intending this podcast episode to be like, let’s pick on a random nonprofit organization that we’ve never heard of before, right? But I do think like there’s some interesting conversation to be had that anyone who manages a website and is thinking about I want to gather this feedback, could maybe have some takeaways.
Asking the Right Questions
Amber: And so that’s my hope with this episode. And so I wonder if we want to start with, do we think the questions that they’re asking in the format of their survey, do we think these are good questions to ask people?
Steve: Yeah, they may be good questions, but they may not be the best questions.
Amber: Why do you say that?
Steve: Because I, I don’t think that they necessarily get to the heart of some of the issues. That, like, there’s really no mention of, do you use a keyboard to, to navigate this website? I mean, they even, they under the assistive technology question that Chris read there actually is a sub subsection there about, you know, what, what is an assistive technology and they don’t even list the keyboard on there.
And, which is probably one of the most utilized assistive technologies, right?
Amber: Yeah. It says assistive technology includes using a screen magnifier or zoom functions, screen reader, voice recognition software, or switch devices. So, I mean, they are open to, we might have users that use more than just a screen reader. So I think that’s kind of cool, but…
Chris: Well, there’s other, another thing I observed is one, the second question is how easy or difficult is it to click on buttons on our website? Which I mean, I’m not here to argue semantics, but that implies the use of a mouse. Versus having a broader question of, you know, how easy or difficult is it for you to interact with our website or our user experience, right?
Amber: You know, I think it’s interesting and I would, I wish I had a good resource on this right now, which I can’t think of one, but I remember in college I took like a statistics class and then a couple of my, like, psychology classes, we talked a lot about doing like surveys or actual research with people.
And, you know, it’s like arguing semantics, but semantics actually really, really matter. And so, for example, on that, do you use assistive technologies to access information on our website? That is a yes or no question.
Steve: Right.
Amber: So somebody could just write: “Yes.”
But what you haven’t said is, what assistive technologies are you using to access information on our website? Which would actually be much more helpful for the person who’s looking at the data. And so I do actually think like thinking about wording of questions when you want to survey your users is really important.
Steve: Well, I mean, like when you talk about the format too, it’s like. This form, how are they going to quantify the responses from, from this, right? Like, are you going to read through each one? And then somebody else is going to like quantify at a, at a priority level. Like, you know, I’m thinking programmatically, how do you quantify the priority of what the feedback that you’re giving? If it’s all these open ended text area fields.
Amber: This is a qualitative survey. It’s not a quantitative survey. So you’d like to see, Steve, if we were doing something like this, you’d want to see at least one or two multiple choice or..
Steve: Stars or yeah…
Amber: Like some sort of, where you have to pick from a list.
Steve: Yeah. Yeah. Yeah. Totally.
Amber: Yeah, like drop downs or whatever. Yeah.
Chris: Well, what’s, what’s interesting to me and what I would probably advise if I were in a position to, like review this survey before it goes out to the public is, I think that a lot of these… Maybe not all of them, but a lot of these are useful qualitative questions to ask. But one thing that I really like that we do on our surveys some of the time and just other web forms in general is you will start with a quantifiable input. So like like, dislike, rate on a scale, right?
And then depending on What the user puts in, a qualitative question that is optional appears based on the user’s input. So if a user puts yes, the text on your website is difficult to read, let’s say that was a yes, no question. Then if they say yes, there’s a question that pops up that’s optional that says, could you provide us with more information if you’re willing?
Like what pages, what text, right? Maybe even a screenshot upload, right? If they’re able to do screenshots, I think all of that would be, a good pivot is to have a mixture of the different types of data.
Amber: Yeah, well, I think, I mean, that was my initial, like, first gut when I saw this is I was like, all these questions are required?
And I know it doesn’t seem like a lot. What did you say? I already forgot what number you said. But it, I don’t know, like, it, it does kind of seem like a lot, right? They might get more…
Steve: I mean, they’re going to bounce. I mean, it’s, it’s a, yeah, it’s a cognitive, cognitive overload a little bit like…
Amber: Well, and they all have text areas. So I think that that contributes to it a little bit, which makes sense. You want someone to be able to give you a paragraph, right? It should be a text area. But I can’t just see at a glance, this is what I’m going to be asked. And it’s like, oh, they have kind of large text areas too. Like they’re relatively tall.
And so it almost gives you the impression I’m supposed to write a lot. Ooh, this seems like a lot of work. I think I’m going to go away.
Whereas being able to pick things right from a drop down or choose a rating of one to five, like, or say yes, no. Okay, I look at that and I’m like, I can do that real fast. And I can get through 12 questions like that real fast.
Steve: Right. I mean, the text field’s probably fine, like Chris said, but you gotta qualify showing it to him first, and maybe don’t even make it required even then.
Amber: So there is something else that is interesting about this website that colors my reaction to the survey. But before we dive into that, we are going to take a quick commercial break and we’ll be right back and we’re going to talk about what that is.
Brought to you by Accessibility Checker
Steve: This episode of Accessibility Craft is sponsored by Equalize Digital Accessibility Checker, the WordPress plugin that helps you find accessibility problems before you hit publish. Thousands of businesses, non profits, universities, and government agencies around the world trust Accessibility Checker to help their teams find, fix, and prevent accessibility problems on an ongoing basis.
New to accessibility? Equalize Digital Accessibility Checker is here to teach you every step of the way. Whether you’re a content creator or a developer, our detailed documentation guides you through fixing accessibility issues. Never lose track of accessibility again with real time scans each time you save, powerful reports inside the WordPress dashboard, and a front end view to help you track down hard to find issues.
Scan unlimited posts and pages with Accessibility Checker free. Upgrade to Accessibility Checker Pro to scan your website in bulk, whether it has 10 pages or 10,000. Download Accessibility Checker today at EqualizeDigital.com/Accessibility-Checker. Use coupon code AccessibilityCraft to save 10% on any plan.
The Importance of Due Diligence Before a Survey
Amber: All right, so. Who wants to share the, I don’t know, what do we call it, the hook here about this survey?
Chris: Well, I was I was just out of idle curiosity during the commercial break trying out some keyboard actions on that website. And let’s just say it left a lot to be desired. Like their menu didn’t expand for me at all, as I was tabbing through. Nothing highlighted. So, I’m, I’m, I have a, I have a guess. I have an inkling of what you’re going to say.
Amber: Of a guess, yeah, yeah, yeah. Yeah, so, so of course, me being me. It’s so hard once you’ve learned about accessibility to not like see it or want to check it. And like, you know, I don’t know, that’s just like the QA person inside of me.
Some of it I just saw instantly, which is, oh, they have orange buttons with white text, which as much as we love Chris’s NASA t shirt that he’s wearing right now, we all know that that fails color contrast. So I don’t even have to use a checker just to be like, oh, you have a contrast failure right there in your header on your donate button.
Or some of your text, you know, maybe in some other spots down lower on the page there’s a light blue. So I, I could see that without doing anything, but I’m always curious. And so I also tabbed through that website. I ran it through our AC offsite and I ran it through WAVE. And it is definitely not an accessible website. Like the navigation menu doesn’t even function. There’s empty links. There’s a lot of stuff.
Steve: Yeah, so for, for a little bit of context for listeners, it, the website has a hamburger menu that is, that’s visible all the time. It doesn’t ever go to a normal navigation menu and the hamburger toggle is, it’s a div, it’s not a button, so it, it never gets focused.
Amber: It has like a JavaScript on click handler or something, right?
Steve: Yeah, yeah, so it never gets focused by the keyboard. Now, it does, like if you keep tabbing, you will go through the links. But like, you have no idea where you’re at, like…
Amber: Because they don’t become visible…
Steve: Right, they don’t become visible. So, so my question, Amber, is how did you find this survey?
Like…
Amber: Oh, oh yeah. That’s the sort of, so when I’m looking for podcast topics, I frequently will just go to Google and type web accessibility, and then I’ll switch to the news filter. And then I filter it by like past month, and sometimes that comes from news sites, but depending on the kind of site, Google might sometimes rank them in news, and so it’ll pull up other things that were randomly published. So this blog, this is surveys in a blog post on this organization’s website, and it was published on January 16th, so it came up as like, new content related to web accessibility in the news section of Google search.
And I was like, oh…
Steve: So you accessed this direct page from Google, not from the website.
Amber: Correct. Yes.
Steve: Yeah. So my question is they’re trying to get, you know, people that require assistive technologies to fill out the survey. How do they get to the survey?
Amber: Yeah. I mean, we don’t know. It’s possible they have an email list and they emailed it out because it’s on their blog too. So maybe they think people who read our blog. I’m assuming that if you make a survey like this, you’re emailing it out to people in your email list. Maybe they have donors or subscribers. They have a subscribe call to action in the header also. But, I, I’m going back to question number two. How easy or difficult is it to click buttons on our website? And when I’m thinking about this survey, like, my instant reaction is: Why do you need users to tell you that?
Steve: Yeah, but what if it’s not a button and it should be?
Chris: Well, I, my, my guess, and I have no factual information to back this up, but my guess is, is this survey was put together by, let’s be real, probably a very well meaning person to try to get some sort of grasp on things about their website that they would need to improve. But not, but that, that person that wrote the survey probably has no direct experience with using alternative means of navigating the web and interacting with it. Like they’ve probably never used a screen reader. They probably never used a keyboard. And so they’re, not, they don’t understand.
Amber: You know, I totally get that. I have a hard time though, because I’m a little bit like, if you’ve read anything about accessibility, you, it’s very hard not to stumble upon WAVE.
Or, or any, I mean, AccessiBe runs ads for their accessibility scanning tool, which will alert you to things like empty links and buttons. Not like you should use that tool, I’m not saying that, but I’m just like. So I am a little like this is, this is kind of where I was hoping to go a little bit, because when I look at the problems that are so obvious on their website that you don’t have to be an expert to use one of these testing tools and find. Like our Accessibility Checker plugin, the WAVE browser extension would find these problems. And this is a WordPress website. I feel like they’re trying to survey users too soon.
Steve: Yeah, yeah.
Amber: Like, why do you need a user to tell you that the button in your navigation menu doesn’t work? Why haven’t you tried to figure that out yourself?
Chris: Well, even beyond that, the users that are most likely to actually have the capability of completing this survey, if they’re going to discover it through the website and not from basically getting a direct email or text with a link, they’re going to have skewed data. Because the people who are going to be able to reach this survey probably have less of a need to use assistive technology, demonstrably than others might.
Amber: Well, I do think the actual survey itself is accessible.
Chris: Yes.
Amber: They’re using Ninja forms.
Chris: You can access the navigation menu to get to the news feed to find the survey.
Amber: Yeah, assuming they’re coming in that way. But my assumption is that’s not like, if I made a survey, like we made a survey for our users, we emailed it multiple times to people and we’re like, please take this. So that’s my assumption. I don’t think they’re making it and just hoping random people will stumble upon it. I could be wrong. Maybe they are.
Steve: But you’re right. I mean addressing the obvious critical issues first is probably, probably best. I mean, there’s definitely some low hanging accessibility fruit here that, that, that can be fixed quite easily.
Amber: Yeah, there’s, there’s sort of an interesting article on the Nielsen Norman Group, which we’ve talked about before. They’re really well known company that specializes in usability and UX. And they had an, they have an article from like 2023. About user feedback request five guidelines and one of the things that they talk about in there is timing. Like when do you ask for feedback can be critical to when you actually get, get it and whether you’ll people will want to engage with it. And for me, I sort of feel like it’s, if you haven’t done at least some accessibility work on your website to the best of your abilities, then it’s sort of disrespectful and a waste of a disabled user’s time to ask them to give you feedback.
Steve: Well, there’s two, like if, if that survey is coming from a link from, you know, some email communication or something. So it’s a direct link to the survey and how, how are they ensuring that these users have actually used the website? I mean, there could be ways to present a survey to somebody after so many interactions, right?
Amber: Hmm. Yeah, or even like, they, this would be an interesting kind of survey. You know how you can do things on exit intent?
Chris: Yeah.
Amber: I wonder if you get a bounce. Like, somebody comes and they’re only there for a few minutes or seconds, and then they try to leave. Maybe that would be an ideal time to survey and be like… So, like, start with one question, is our website working for you, yes or no?
And if they say no, then you could pop them over to be like, can you give us more information about that? Right? To try and figure out why are they just leaving? And then maybe you’d get a better response rate.
Steve: Yeah, I think so.
Amber: Or or like, I know in Hot Jar, you can see what they call rage clicks. Where like, people are like clicking a bunch of something they think should work or do something and it’s not. Like, maybe you could use rage clicks as a trigger for, okay, this might, there might be a problem on our website. We need to show them a survey. Yeah.
Chris: I do think that there is this propensity at times to, because it gives the feeling of doing the good work to just ask for information or ask for feedback, right? But I think it’s kind of a weird situation when you’re like, or I think that what I was trying to communicate, sorry, is that I think that there’s this propensity to ask users with disabilities to do their own testing and validation and to tell you if you’re doing a good job, versus using scanning tools or hiring a professional to help you. Or, or in the, in the instance of a nonprofit, maybe finding a volunteer, right? To, to help with that who’s willing to give their time. But just kind of trying to outsource your accessibility audit to your users, I think is kind of.
Steve: Yeah.
Chris: It’s not good. And again, I don’t, I’m not here to to crap on this organization. Like, I, I’m guessing they probably mean well, but they’re, they’re doing a lot of the things that in my pattern recognition, I see, I see organizations that don’t really get accessibility do.
We Speculate on Possible Reasons for a “Premature” Usability Survey
Steve: Yeah. Yeah, I would say too, like it, I don’t know how this organization is set up, but it, it may be like somebody in their organization is trying to be an accessibility advocate, right. And, and they’re trying to maybe create some kind of data or proof that accessibility should be prioritized with, within their shareholders or their board or whatever, right. That may be the intent of this form.
Amber: That’s a good thought. I didn’t really think about that. Like maybe they want to do more and they want to hire a developer to fix their navigation menu. And someone else is like, we don’t have budget for that. And so they’re trying to prove that it’s necessary.
Steve: Yeah.
Amber: Oh, that’s, that’s a really interesting thought that might justify having this survey earlier.
In the process.
Chris: The other, to flip that on its head. I’ve served on boards in public sector And one thing that boards will sometimes do is direct the director of the non profit entity or the charter school or whatever it is, to put out surveys about certain information that the board is curious about. So this also could have been a board directive. And a somewhat ill placed one in the ideal order of operations.
But the point is we don’t know, but I think the deeper point that we’re all trying to make here is that if you’re going to ask for feedback, do a little bit of the basics first, otherwise you’re kind of in a, kind of in a weird situation. I I’m trying to think of like a, a re a real world, non digital comparison of what that would be. I don’t know.
Multiple Speakers: (Unintelligible Cross-Talk)
Amber: Having a restaurant that, that only has stairs
Steve: Oh, yeah.
Amber: And then you’re serving users and you’re like, how easy is it for you to get in… Or users, customers! How easy is it for you to get into our restaurant?
Steve: And they’re outside. They’re like, I’m not even in your restaurant. I can’t get in.
Amber: They’re like, have you looked at your door? That’s, that’s kind of how I felt about how easy is it to click buttons on our website?
Steve: Well, they’re not buttons.
Our Best Parting Advice for User Surveys
Amber: So, so I think this is a good closing point. Maybe we could each say, what do we think our top recommendations are for organizations that want to survey their users on the accessibility of their website. Do we have any ideas about either, I think, and we can each approach it whatever way makes sense to us, either the questions we would include, things we would do first, how we would send out the survey. Like, how would we advise organizations to gather information?
Who wants to go first?
Chris: I’ll, I’ll go first.
I think providing more context around why you’re doing the survey. I think we’ve gone in deep enough on, like, how to make this survey more effective, as a, as a, as a tool and what types of questions to ask qualitative versus quantitative. But I think more transparency around why this survey is being put out beyond the two sentences that are at the top.
Like, was it an internal staff member? Was it a board member? What are the organizational concerns they’re trying to address? Is it legal or some sort of compliance thing? Right? Like and I think that that can help the type of response and the reaction that you get if you just give people more information about why you’re asking them these questions. Beyond just a generic, we want to make sure our site works well for you.
There has to be a deeper reason I’m guessing, beyond that. Especially if you’re in the nonprofit sector.
Amber: Well, we ran a survey recently for our roadmap planning, and we even gave a very specific date also. And like we said, Amber, Chris, and Steve are going to meet on this day in January. I can’t remember what it was.
We’ll pretend it was January 9th. January 9th. And we would like you to submit responses before then because we will be discussing them in this meeting and using them to do XYZ, right? Versus sort of an open ended, like you’re saying, Chris, we want to make it better blog post. And then maybe it doesn’t feel like you’re asking for feedback that’ll just go into a black hole.
What’s your best advice Steve?
Steve: I think my best advice would be to I mean first let’s let’s fix the obvious things with the website. Like you said, it’s it’s a little disingenuous to ask these things when you haven’t taken any measures yourself to try to fix accessibility issues that can be exposed with literally the click of a button, like with the WAVE extension. I mean, this is a WordPress website. You could install the Accessibility Checker as well.
Amber: Free one.
Steve: Yeah. The free, free Accessibility Checker. Yeah. As far as the survey, yeah, I would kind of just reiterate that you know, make it easy for them to respond. Don’t put all the onus on the data on the user.
Do some of that work yourself and then present it to them in an easy way for them to click either, you know, radio buttons or checkboxes or star ratings. You know, just to make it much easier for them to give that information and, and much more likely that they will complete that form and give the information. I’d probably think about the questions you’re asking a little bit better, be a little bit more direct.
Maybe, you know, there is some logic to the questions. If a question is answered this way, then you present something else to go a little bit deeper. I would probably provide an alternative format to, to the online portion. Especially if your website’s not accessible.
Chris: That’s a great point.
Amber: What do you mean by that? Like a paper survey?
Steve: Well, I mean, maybe. Maybe, maybe a, a PDF. Maybe some other service that is accessible.
Amber: Oh, like not hosting it on your website, if you know your website has accessibility problems. Go do it on, I’m going to pretend SurveyMonkey is accessible. It could be, I don’t know, I’ve never. Like that? That’s what you mean?
Steve: And I’d probably be a little open with it. Like I definitely would do something that is programmatically quantitative to where like the results can, can be presented maybe even transparency too. Like, if you, you know, fill out the survey to learn about how our users feel about accessibility and when they fill it out, then you actually present some data at the end of that survey.
And it just says, just says like, thank you, your entry has been submitted with, with no feedback. Nothing like, you know, we plan to collect this data and speak about it with our internal board. And then we’re going to provide our findings here and then we’re, you know, we’re email our list with, you know, what our plan is next to address these accessibility issues. So like, right now it’s like all the work’s on me and I get nothing for it.
Amber: Yeah. So I, I agree with both of you. I would, I would make it shorter. I would not ask obvious questions, like do our buttons work or not? But you know, the one thing that you guys haven’t mentioned that I do think has been really helpful for us every time we’ve done surveys is we always put an optional email address field.
And, and it’ll literally the label will be optional email address. Like I’ll write the word optional and then we’ll have a description for the field, which says, if you would like us to follow up with you about your response, enter your email address here. And 50 percent or more of people will provide their email address and that can be really helpful because I mean, we’ve done surveys where they’ve said it would be great if this existed.
And, and I was like, oh, it does. They just don’t know. So I like emailed them and I’m like, hey, here’s how you do XYZ. And they’re like, Oh, wow. That’s so great. I didn’t know. Right.
Or we’ve also, I’ve also reached out to people because they’ve written something that I didn’t really understand and I wanted to understand better. And so I would be able to then get more information.
So I, that would be my best advice to wrap up on top of what you all said is, anonymous surveys are fabulous, but some people will want to provide their information and giving them a way to do that or inviting them to do that. Obviously, they can put whatever they want in an open ended text field, but providing a thing that says, we will follow up if you give us your email address, or we may, if you don’t want to guarantee you will if you think your volume will be too high, I think could be really helpful.
I, I would say overall, I think they have a good intent, but I would, if I was consulting with them, I would advise them to do some work first before they. They probably already sent this out to their email list, but.
Well, hopefully this has been a good conversation about surveying users. I don’t think we’ve ever talked about that before. And got a nice mango seltzer to go with it for our Friday afternoon.
Steve: Very good. Great talking. Cheers guys.
Amber: Bye.
Chris: Thanks for listening to Accessibility Craft. If you enjoyed this episode, please subscribe in your podcast app to get notified when future episodes release.
You can find Accessibility Craft on Apple Podcasts, Spotify, and more. And if building accessibility awareness is important to you, please consider rating Accessibility Craft five stars on Apple Podcasts. Accessibility Craft is produced by Equalize Digital and hosted by Amber Hinds, Chris Hinds, and Steve Jones.
Steve Jones composed our theme music. Learn how we help make thousands of WordPress websites more accessible at EqualizeDigital.com.