Checking Accessibility On Your CI Pipeline With @axe-core/cli

I said I would write more technical output on my blog, as well as just about my journey, though I feel this post can combine both – but with more of a lean towards the technical. Mostly because checking accessibility on my CI Pipeline is something I struggled to find a useful tutorial on.

A CI (continuous integration) pipeline is a set of automated processes that checks code that is being merged in – the theory being that a solid set of automated checks will stop most breaking changes from being merged, and engineers can happily code away, continually releasing, secure in the knowledge that they are unlikely to break anything.

There are different checks that you can do – ensuring that your testing suite doesn’t fail, ensuring that you are not slowing down your loading speeds with Lighthouse checks, checking for security issues – but here I’m writing about checking for accessibility issues.

In case you don’t know, accessibility is ensuring that your website works for all users – especially those who are visually-impaired – an overview of the requirements can be found here.

Accessibility is core to my soul as a human, yet it is easy to overlook when creating a feature under the pressure of a deadline, or if you are just playing around with writing your own blog, and want to perfect the design. Hence why checking accessibility on my CI pipeline is a feature I feel I need for my personal websites, including this one. Alas – it isn’t a panacea, but it will pick up some crucial things such as colour contrast, and also some lesser-known points – as we shall see. Well, you will if you keep reading.

Taking An Axe To My Problems

The more personal journey-based aspect to this, is that I don’t have any exposure to managing CI pipelines at work, as all that area is taken care of by a centralised platform team, so adding CI pipelines to my own personal projects, such as this blog written with NextJS/React/TypeScript is a way for me to gain exposure and insight, rather than it being a complete unknown to me.

So inside my project, I have a .github folder, inside that I have a workflows folder, and inside that a pull_request_audit.yml file.

When I raise a pull request that includes this file, the checks inside will run.

name: CI Tests For James Winfield
on:
  pull_request:
    branches: [main]
jobs:
  axe:
    runs-on: ubuntu-latest
    env:
      WORDPRESS_API_URL: https://blog.jameswinfield.co.uk/graphql
      GITHUB_API_SECRET: ${{ secrets.PIPELINE_GITHUB_API_SECRET }}
    steps:
      - uses: actions/checkout@v3
      - name: Use Node.js 18.x
        uses: actions/setup-node@v1
        with:
          node-version: 18.x
      - run: yarn install
      - run: yarn run build
      - run: yarn dev & npx wait-on http://localhost:3000
      - name: Run axe
        run: |
          yarn add @axe-core/cli
          npx axe http://localhost:3000 --exit

You’ll need to make sure you provide any secrets that your environment will need to run on the pipeline – for example, with this being a headless WordPress blog, I have to provide the WORDPRESS_API_URL so it knows where to find my graphQL data, plus the GITHUB_API_SECRET.

At the time of writing, Node 18 is the version that is working for me – but I assume in the future that I’ll need Node 19, Node 20, Node 528, etc. Also, I’m using yarn, but if you are using npm/pnpm or some other package manager, make sure your commands are aligned with those on your package.json file.

Chopping The Tree Down

My first successful attempt at running this, led to:

Two issues. Firstly the violation of aria-roledescription.

This was quite easy to find and fix:

According to the documentation, aria-roledescription should used on elements with an implicit or explicit role value. I admit to not being entirely clear here (some irony that accessibility documentation is not very linguistically accessible) but I figured that I didn’t need this to explicitly be a subtitle, so I removed it.

Time to push my code, and for the pipeline checks to run again:

Woohoo – I have one less violation.

This one was a little easier to understand – a lack of colour contrast in my footer.

Yikes – pretty clear even to my reasonably perfect vision that the contrast could be improved – and this is the perfect example of why we should use accessibility tools somewhere along the line – at least running Lighthouse checks locally if you are not comfortable with using CI pipelines, for example. But I like the automated and enforced nature of CI pipelines.

I switched my footer background to be a dark blue, and the font to be white, to match the rest of the fairly quickly thrown together theme, pushed my changes, created a PR and voila!

No violations found! And we can all live happily ever after. Can’t we?

Chromedriver Comes Crashing Down

A month or so later, my accessibility checks started failing:

What. Is. This.

I was stumped. This was happening on my other sites too, so it wasn’t related to anything specific on this website. ChatGPT kept sending me into endless rabbit holes, I couldn’t find any tutorials that had any code that would solve the Chromedriver issue, there was nothing on Stack Overflow, there was nothing relevant on the issues section of axe-core’s Github pages (though definitely mentions of the Chromedriver issue).

I gave up, and switched off the requirement for the accessibility check to pass on my pipeline. Bad developer. Eventually I had another crack, I braved the notion of opening a question on Stack Overflow (downvoted of course), someone pointed me in the direction of a Github post – the solution didn’t work, but there was enough of a structure to give me something to work on, and after around another oooooh 20 pushes and much research, I eventually found a set of commands that works:

name: CI Tests
on:
  pull_request:
    branches: [main]
jobs:
  axe:
    runs-on: ubuntu-latest
    env:
      WORDPRESS_API_URL: MY_WORDPRESS_API_URL
      GITHUB_API_SECRET: MY_GITHUB_API_SECRET
    steps:
      - uses: actions/checkout@v3
      - name: Use Node.js 18.x
        uses: actions/setup-node@v1
        with:
          node-version: 18.x
      - run: yarn install
      - name: Get Chromium version 🌐
        run: |
          CHROMIUM_VERSION=$(wget -qO- https://googlechromelabs.github.io/chrome-for-testing/LATEST_RELEASE_STABLE | cut -d. -f1)
          echo "Chromium version: $CHROMIUM_VERSION"
          echo "CHROMIUM_VERSION=$CHROMIUM_VERSION" >> $GITHUB_ENV
      - name: Setup Chrome 🌐
        id: setup-chrome
        uses: browser-actions/setup-chrome@v1
        with:
          chrome-version: ${{ env.CHROMIUM_VERSION }}
      - name: Install chromedriver 🚗
        run: |
          echo "Installing chromedriver version: $CHROMIUM_VERSION"
          yarn add chromedriver@$CHROMIUM_VERSION
          echo "chromedriver version: $(chromedriver --version)"
      - run: yarn run build
      - run: yarn dev & npx wait-on http://localhost:3000
      - name: Run axe
        run: |
          yarn add @axe-core/cli
          CHROMIUM_VERSION=$(google-chrome --version | cut -d' ' -f3 | cut -d'.' -f1)
          yarn add chromedriver@$CHROMIUM_VERSION
          npx axe --chromedriver-path $(yarn bin)/chromedriver http://localhost:3000/ --load-delay=1500 --exit

I hope this works for you – and helps improve accessibility across the web.