Hi, JSConf! I am so excited to be here. I
gave a talk last year in the B track of accessibility of web components.
If you caught that talk, you may remember a secret thing that happened at the
end which resulted in real life tacos. To admit, though, I actually… I’ve
got to say I like pizza more than tacos. If I could choose, I would always choose
pizza every time. I really wanted to work pizza into this talk somehow,
and at the risk of sounding cheesy… We are going to make this happen.
So hold on a second. Hi. I’d like to order a pizza. Yeah. Let’s
do… One cheese and one pepperoni. Yeah. You’re going to bring it to Marcy Sutton.
JSConf in track A. Yeah. You have 30 minutes. Thanks. Bye. Okay. So…
Now we can get started. In the time it takes for this pizza to get here, I’m going
to tell you about something I’m really passionate about. Which is accessibility.
We’re going to talk about automated accessibility testing. Which is
a way that you can actually work accessibility into your workflow. I’m Marcy
Sutton, I’m a developer in Seattle. I currently work at a company called
Substantial, and I want to give them a shoutout, because I went to Substantial
to learn about git and pragmatic software practices, and now I’m
giving a talk at JSConf. So I’m pretty stoked that I get to work there and
work on this stuff. In July, I’m moving to Adobe, so I’m pretty stoked about
that. I’m an Angular core team member, I’m devoted to ARIA, the accessibility
module, and since July I’ve been working on Angular. We’ll talk a little
bit about it, but the focus today is obviously automated accessibility testing.
My slides are on Github. There’s a shortcut for my website, Marcy Sutton.com/JSConf2015.
So accessibility — the answer to a more inclusive web. It’s the
way that we can build websites and mobile apps and experiences that work
for more people. Because a fifth of the population on the planet
has some kind of disability. So if we don’t — if we keep not addressing
accessibility, we’re leaving out a ton of people. Pizza is also always the answer.
Just saying. I’m going to gloss over some accessibility basics, because
we’re focusing on something really technical. Automated testing. But you
should become familiar with accessibility basics. I have them listed in
my slides and there’s a link to more principles if you want to read more.
But some things you should be considering if you’re a web developer are
text alternatives. Including alt text for images, form labels, offscreen text
if you have infographics and things like that, document structure and hierarchy,
which we’ll touch on a little bit today in talking about headings,
HTML semantics, which you should definitely be using. Use real buttons and
so on. Keyboard interactivity, making sure if you’re creating custom controls
that they’re reachable and operable from the keyboard and they have adequate
focus styles. Color contrast. That’s a huge population of
people. As well as keyboard interactivity. Not just blind people or people
with deafness or hard of hearing — these visual disabilities are a
little more subtle, and they affect huge populations of people. So color contrast
is in that category. And visual contrast in general, which even my slides
— this projector looks a little different than my computer screen. So my slides
need adequate contrast. And then lastly, if you’re building single page
apps or client rendered applications, you should be familiar with
focus management. It means that you’re guiding the user’s keyboard
focus around your apps, so that if you’re opening dialogues or bottom
sheets or other components, that their focus isn’t being dropped. That you’re
actually curating their focus experience so that you can notify them when
things are opening and things are happening. And I have some resources at the
end if you want to learn more about that. But there was a time when I didn’t
know accessibility. Like, you have to start somewhere. And so it’s important to recognize that you’re
not going to be perfect at this right away. Jake the Dog from adventure time
says — dude, sucking at something is the first step towards becoming
sort of good at something. And that’s okay. You know? It’s not something
that’s going to be perfect all the time. But every little bit of accessibility
that you contribute is so appreciated and so needed that you can just
learn more as you go. And it’s a huge win, any time there’s something for accessibility
that we’re including. But we’re developers. This is a JavaScript
conference. So we’re going to let the tooling that we’re already familiar with
— using JavaScript task runners and tools that we love to work with, because
they’re technical and geeky — we can use this tooling to do some heavy lifting
for us. Help us identify accessibility issues in our apps. The low
hanging fruit stuff that you don’t need to pay someone to find, if you can automate
that. You can free up people’s time to test things that are a little
bit more nuanced and more subjective. So we’re going to talk about manual
testing. You know, where do I start when I audit a website for accessibility? And then we’ll move into what I’m calling
sort of automated testing. Not looking at the rendered source code, but using
some tools to help you identify issues. And then lastly, we’ll talk about
what you really came to hear about, which is definitely automated testing. And
how you can incorporate accessibility testing — so, for example,
if you could prevent a broken build from going out into the wild. Automating that,
so it can withhold a deployment of some kind. But the elephant in the room that I have to
mention to you, because we’re talking about automated accessibility testing,
is that it is no substitute for real user feedback. Including people with
disabilities. You want to have people testing your apps so that they can
tell you when something really sucks. Like Peter Griffin saying what really
grinds his gears. Because that feedback is so valuable. So that should be
part of your strategy as well. And even getting to accessibility earlier in the
design process, too. Those are all very important things. But we’re here
to talk about automated testing. Which is an important thing to know how to
do. So the first thing that I do, when I look
at a website to figure out what the accessibility situation is — I just tab through
the page with the keyboard. So we’re going to do that real quick. I’m
going to show you gov.uk, which has pretty good focus styles, solid blocks of
color, the text links are either bolded with blue text or bold with white text,
and if we start tabbing through the page, the focus indicator is really obvious.
It’s orange color on whatever the other background color is. And it actually
really shows you effectively where you are on a web page. It has been pointed out to me that this is
technically not WCAG AA compliant. And that’s because there isn’t quite enough
visual contrast to meet that standard. But what they could do to fix it
is either widen the visual contrast between the foreground and background colors,
or you could make the text bigger. So you have some things that you can
do to fix this. But compared to most websites, gov.uk is awesome, and it’s
winning the focus style… Like, they do it the best of any site I’ve seen.
So they deserve a pat on the back. Okay. So that’s the first thing I do. The
next thing I do — which I’m not going to spend a whole lot of time on — but
you should be demoing in screen readers. I’m going to show you SoundCloud
in Safari with VoiceOver on the Mac. But it’s not the most used screen reader.
JAWS is the most used. NVDA and Firefox are well used. There’s also mobile
screen readers. IOS Safari, TalkBack with Android is great. But you should
be checking your stuff in a screen reader so you know what’s going on
in your app. So we’re going to demo… Oops, wrong window. So I’m going to hit command F5 and fire up
VoiceOver. VoiceOver on Safari. Stream on SoundCloud.
Visit link. So you see how the SoundCloud link… It is
just a graphic. There’s no text saying SoundCloud, but when I use the screen
reader, SoundCloud has done a great job in adding alternative text to their
images. When I tab through it with a screen reader, even though it’s a graphic
button, I get some text, because it says SoundCloud on it. Visiting link. Link. Link. Upload. Banner
seven items. Marcy_SSS pop-up button. So this is a pop-up little toggle thing. If
I hit enter, it opens the pop-up and sends my focus into it. Now I’m
in this list of child links. You are currently in a list. Trick with VoiceOver — mute it. So we’re
in this list. And to get out of it, I hit escape. Pop-up button. Banner seven items. VoiceOver
off. So that’s just a really quick demo with VoiceOver,
to show you what it’s like to use it. And again, you just turn it
on with command F5 on the Mac, and you can go check it out. And I will say Safari
works a little better than Chrome. But it’s a good thing to do every
now and then. So that’s one of the first things I do. Now I want to get into
sort of automated testing, using a few tools that I think are useful. Something
that you should be checking for are headings and semantic structure. So the
web was created for documents. We’re creating applications. It doesn’t mean
that because you’re creating an application you get to throw out the semantic
structure and not care about it. Because a user with a screen reader can use
your heading structure to get an overall picture of what is on your site. They
can also navigate by headings and other landmarks. So this Firefox web developer
tool bar I loved, because it can show you the heading structure. So
if you install the tool bar, go to information, and then view document outline,
you can see the heading structure on your page. I still use this. It’s probably
the only feature in this tool bar I still use. But for my Tumblr, accessibility
wins, I could go and look at it and it’ll show you when everything is all
aligned. So next I want to tell you about the Chrome
accessibility developer tools. It’s a must have for any well developer these
days. It’s great for checking the overall page accessibility for any website.
Especially in active development. It catches kind of the low hanging
fruit of things that you — like missing labels and things. So let’s go
check out an audit. Okay. So… I’m going to pick on CNN today. Let’s go see
what crusty accessibility things they have going on. So I’m going to open up
the dev tools. And I have the extension installed. So under
audits, I get an accessibility audit, which I can go and run, and it will
go and it will look at their site, analyze it, run tests against it, and it will
report back to you what’s wrong with it. So they are missing a bunch of form
labels. If I drill down into it, I can look at the specific node that is a
problem. I can review it in the elements panel, and go over here in the element
inspector. You get a couple little add-ons. Actually, with the extension,
you get this accessibility properties panel. And it will show you some
of the accessibility information. I’m using Chrome Canary, which actually has
a dev tools experiment. If you’re curious, I can send you the information later.
Because it’s sort of hard to enable. But the deal is that this extension
will be moved into the dev tools at some point. And Google is working on that
right now. But if you have it enabled in Chrome Canary, you get this secret
panel of extra information of the accessibility node. And this will tell
you what role it has, if it has any ARIA attributes, and so on. That highlights an important concept that
I want you to know. Which is about the accessibility tree. The accessibility
tree is something created by browsers. It is a parallel structure to the
DOM. And it has in it semantic information that screen readers and other
assistive technologies can use to present information without the performance
implications of everything else in the DOM. So the accessibility tree you can
check a couple different ways. If you go to Chrome://accessibility, you can
look at a raw dump of the accessibility tree of every tab that you have
open. Which you can imagine — it’s turned off by default. Because that would
be a pretty big performance hit. I have a post in my slides from the Paciello
group with more on the accessibility tree, including their tool,
called aViewer, which is really helpful for that. But when I first got into
the Chrome accessibility developer tools I found on Github that all the library
code is out there under the Google Chrome organization on Github, down
to every audit rule as a JavaScript file. And I didn’t quite know what those were for.
I felt like Finn from Adventure Time going — what do you do with those? Well,
that’s where we get into definitely automated testing. I’m going to
start by showing you a tool called a11y. It’s a numeronym for accessibility.
You take out all the numbers and add 11 in the middle. You see me tweeting that
all the time because it’s shorter. But they made a really great NPM module. You
can install it with a11y, and you get this tool where you can call the same
audit that we ran in the browser from the command line. So I’m going to show
you this. Instead of pulling it into a JavaScript file, we’re just going to
run it from the command line. So I’m going to type a11y cnn.com and then we’re
going to wait and hope that it resolves. It takes a second. It’s a little
bit longer than the browser extension. But this is using phantom.js, which
is a headless browser. So we’re not seeing the browser open. This is running
on the command line, using phantom, hitting their site, and returning
an audit back to us. Hopefully this will work. Hello, conference Wi-Fi. So the
audit that it gave back to us was the same audit that we got in the browser
extension, except this is on the command line. So I find that this report is
a little bit harder to digest than the browser extension. But you can see that it tells you the same
warnings. It says the controls and media elements should have labels. And then
it points you to a node in the DOM tree. It actually shows you the tree of items,
selectors, that it actually used to point you to that node in the browser.
So this is pretty cool. But I’m not quite sure how I would use this in a project
yet. I will add that there is a grunt task for a11y that could be useful.
Ways that you could integrate automated testing for accessibility. What if you ran it on every save? So that
you could check whether something was inaccessible? It might be a little aggressive.
But you could also run it on every commit. Maybe you wanted to check
that you had a pre-commit hook for git, and you could check whether something
is broken. More likely, you’re going to run it on a deployment. If you push
out a build that’s broken, I’m either not going to let it go forward or I
can use Travis on Github to say — hey, the build is broken and that person broke
it. Which leads us to protractor, the end to end testing framework
for Angular. I’m focusing on this because I worked on it, but the concepts are
similar for any framework. Protractor is a node module that runs on top
of Selenium web driver. It fires up a browser programmatically, you tell it
where to go, and you set up your website so you can run tests against it and
assert that it’s doing the right thing at the right time. I wrote an accessibility
plug-in for protractor that I’ll tell you about a little bit. But protractor would be really useful — it’s
what’s called end to end testing. It’s useful for testing multiple
components on a page together, as opposed to unit testing, which we’ll talk
about in a minute. But some things that you could test for include focus management.
Like if you are writing a single page or client rendered app. You could
check for live updates. Like, say your focus is in an input field, and the
act of typing filter something somewhere else on the screen — you want to
make sure those components are working together. That might be a good thing
to test for with end to end testing. Or color contrast. Maybe you want
to check the color contrast across your whole app. You can do that in this as
well. So to get an example, we’re going to look
at Angular material start, which I mentioned I’ve worked on Angular material,
which is a component library built on material design — build on Angular, using
material design. The Angular material start is just a small repository
you can grab on Github to get started. So I wanted to incorporate the protractor
plugin. It’s pretty simple. It only has one view. It has a list of users
on the left site nav and when you click on one, it opens the specific view in
the primary column on the right. There’s a little share button, and if you
tap that, it will open a bottom sheet, which is kind of like a dialogue, in
that it opens on top of things. But it is a sort of drawer that comes up from
the bottom. It’s kind of a mobile pattern in material design. The idea with this, though, is that it is
an interim element. And so when it opens, for a keyboard user or a screen reader
user, we want to send our focus there. Both so that they are alerted to new
content, but also so their focus is in the general location of this new content.
This would be a perfect thing to write a test for. I want to assert that
the first button in the bottom sheet has focus. So some setup I had to do
for protractor — the config file — this is in the project now, but I had to
add a line to enable the plugins, which come with protractor now, that come
with the Chrome accessibility developer tools. You just have to enable them
in your config. It also supports the Tenon API, an accessibility API. It’s
super awesome, really robust. It requires a subscription and an API key, but
that is worth considering as well. And the other thing I had to do was add a
selenium address in the config file. Then in the package.json, under the dev dependencies,
there’s something called scripts, where you can put any arbitrary hook
you want to run from the command line. For me to run NPM test, I put a binary
from protractor, which is the node module we download with it installed,
and then the config. So once we have this set up, we can write some tests.
The tests in the project — I think they look a little different now, but the
tests for the bottom sheet assumes that you have already navigated to the page
and that you have already clicked on one of the avatars on the left. But this
is a jasmine test, so it starts with a describe statement, just an arbitrary
what we’re testing, and then for each, we have to click on that little share
button to get the bottom sheet open. But we’re setting up the selenium web
driver to be in the state we want to test. So once that happens, we can assert
that it should focus on the first item. And this — there’s an infinite number of
ways that you could write a test. This was the first pass I took at this. And
what I did was I iterated through the buttons in the bottom sheet, and when
it focuses on one, we have a hold of the active element. So I just verify that
the one we’re iterating over is the same one as the active element. You could
do this other ways. But that was how I verified that the focus management piece
was working. So on top of that, that’s a test that I wrote, and that is the
secret to this plugin. Is that you have to write tests that hit different parts
of your app, and then the plugin currently — it doesn’t crawl your whole app.
You have to write some tests, and then if you have the plugin enabled, like
I showed you in the config, it will actually run the same audit tools — or
the audits that we ran in the browser extension, and Adi’s tool. Same library.
It will run those in your markup in protractor. So in material, everything
passed. So I had to make it fail to show you what a failure looks like.
Ironically, the pass and fail messages in the command line are in red and
green, which I’ve heard are a little hard to see for some people. So that’s
a change we have to make. But it also says pass and fail in text, so it’s not
relying on color alone. But that’s an improvement I would like to make.
Which I can imagine — even on a different monitor would be harder to see.
And quickly mention that Tenon API would be another option. If you’re interested
in that, look at the protractor documentation. But to fix those failures that
I made fail on purpose — it has a list of contexts that it iterates over,
and objects, and for each item there’s a button with an icon on it. I just
removed the text from the icon so it would fail, and to fix it, I just add the
text back in. Which you can see highlighted. And then we’re
going to run it on the command line and see if it works. So I have the command
line in my slides, and I’m just typing NPM test. That’s the hook that
we put into package.json. So I have selenium web driver running in another tab
in my terminal. We can see selenium open up the browser, and now I have to go
back to the slide. So the difference between Adi’s tool and this one was Adi’s
used phantom. So it didn’t open another browser. It didn’t move our focus
away. With protractor, it’s actually using web driver to fire up a browser. We
specified Chrome in the config, but you could fire up Firefox, you could run it
in other environments — but what that’s doing is actually running our tests
against that real browser. And then since I fixed the code in the HTML, now it
passes all of the failures that we had before. We can also see that there are — in the actual
material start, there are two tests. I only showed you one of them. But
the actual real project, which I’m running on my computer, that has two tests.
And then we have the plugin enabled, so on top of our tests, it runs the
Chrome accessibility audit. It’s pretty awesome. I’m pretty stoked about that.
So in comparison, unit testing. And actually, this is something I use more
in my day-to-day work as a developer. Unit testing is different in — different
from end to end testing in that you need to write tests for individual
components. Things you could test are — ARIA properties that need to change
with user interaction, keyboard operability, to make sure that tab
index is enabled, and keyboard commands work. Text alternatives, making sure,
for example, if you had an icon button, and you wanted to make sure that it
had a label, or it fell back to a label or something — semantics as well. So
unit testing is a really great way — if you’re not already doing unit testing,
you should be. That’s what I went to learn at my last job. But accessibility
fits in with that so well, because you all of a sudden have test coverage across
your whole app for accessibility. So an example that I use a lot is that I am
the primary contributor to NG ARIA, the accessibility module for Angular.
There are some things I like about it, some things I want to change, but I have
an example of a custom check box, a custom element with web components and custom
HTML now. You can write any tag name that you want. So I have sum-check
box. And then using ng model and ng checked in Angular, I can have it dynamically
add attributes. So what ngARIA is doing is ensuring that the ARIA
checked attribute is on the attribute, as well as the checked attribute.
So if I uncheck the check box, we see ARIA checked change to false and the HTML5
checked attribute go away. And that actually — checked is what NG checked
is adding. But because this isn’t a real check box, it’s
not an input, we need ARIA to tell the accessibility tree in a screen reader
that it’s checked or unchecked. But the one thing we want to look at here
is that this has tab index. So NG model, when it detects a role of check box
— there are certain scenarios where it will add tab index dynamically. And
that is what I want to show you the test for. I’m refactoring NG ARIA right
now, because it’s a bit too liberal with what it adds. So I have tests
that I’m working on locally about which elements it should attach to. And because
Angular has thousands of unit tests, I usually use — well, I will isolate
to the NG ARIA tests. So I validate that it should not attach to native
controls. Buttons — if you put NG click on a button, it doesn’t need tab
index. It should attach to custom inputs, and it should attach to elements with
NG click, like divs. So this is the stuff that NG ARIA — the opportunity
we have to improve Angular applications accessibility. But we need a
little finessing to get this module in a better place. The unit testing is obviously
a huge part of that. Actually, I’ll talk a little bit more about
that, since we have some more time than I thought we would. The unit testing
— every time I go to add a feature to something like NG ARIA or angular material,
I will start with a test. Test driven development — we’re all familiar with
that. But for accessibility you can do that. So when I went to make these
changes to NG ARIA, I would start with the test. But it would fail. It’s doing
things it shouldn’t right now. So I start by writing the test and then I go
back to the code and then I make the changes to get all the tests to pass. There
were actually a lot of features in NG ARIA, so it wasn’t as easy as I thought
it should be. But that should be coming up soon. So today we’ve talked about keyboard and screen
reader testing as a manual way to actually check or diagnose the accessibility
of an app. We looked at sort of automated tools like the Firefox web developer
tool bar and the Chrome accessibility developer tools, which the latter
— the Chrome tool I would highly recommend using. Firefox — the web
developer tool bar is cool. It’s a little bit more dated, and I would say the
heading feature is the only one that I really use. But then in automated testing, we talked about
Adi’s a11y module and protractor. There’s other tools out there
like Quail and Tenon and other things you can use to automate accessibility
testing. There’s more of a culture. I’m here talking to you about it
at JSConf. I never thought this would be a subject you would want to hear
about, but it’s changing. There are more tools and resources for us as developers
to make the world better, make a more accessible web. So it’s really exciting
to see these things start to happen. In case you’re like — whoa, okay. I know
about automated stuff, but I don’t know anything about accessibility. I have
a ton of resources in my slides, including people with disabilities user research.
Because that’s something we should be doing. Conducting user evaluations.
As developers, I always think it’s really cool when I get to go do a different
task than development, like interviewing users. I think that’s really
valuable. I have a couple blog posts about auditing a website for accessibility,
a recent one from smashing magazine on client rendered accessibility.
There’s some more tooling on W3C posts about web accessibility evaluation tools,
the screen reader survey is the closest thing we have to analytics for
accessibility. What do people self-report? What do they use for browsers
and screen readers? And we can look at that and look at our own analytics to tell
— do I really need to support IE8? You can go and check. And then a little bit on what is ARIA, because
I didn’t go into that very much. So that means that we’re at the end.
And there should be something awesome happening now. Beyonce dancing, with,
like, a pizza coming out, I think, is like a win alone. We’re actually
done a little bit early. So I wanted to say… In case you don’t get a slice
of pizza, I have some pizza buttons that I would be happy to give to you.
And you can come and find me at any time, and ask me questions about accessibility.
I’m pretty passionate about it, obviously. Our magic thing is not
happening. Oh. That sucks. (laughter) You know, I had a really good run. Last year
was amazing. And I guess it’s just not happening. What was supposed to happen
was that there was supposed to be a pizza that came out. I actually really
want pizza. I skipped the afternoon snack. So it just… My dreams are
foiled. Your pizza dreams. As a consolation prize, I have an inflatable pizza
slice that we can play with in the pool. I don’t know if you heard about
that. I’m really glad I brought that now, because I’m so let down that the pizza
didn’t show up. Give them an angry phone call. Anyway, thanks for letting me
dawdle here for a minute. It’s been really fun talking to you. So thanks!

2 thoughts on “Marcy Sutton: 30 Minutes or Less: The Magic of Automated Accessibility Testing | JSConf US 2015”

  1. I'm so stoked that this is something people are talking about! My friends at work and I have always pontificated on accessibility, we agree that it's the right thing to do but it's hard to know how to do it properly or make a business case for it, hopefully that will start to change as more people discuss it.

Leave a Reply

Your email address will not be published. Required fields are marked *