While working on Paylooza it was really important to me to have a very well tested application. Paylooza strives to be an affordable, off-the-shelf recurring billing solution for individuals and small businesses and because we are dealing with transactions it is important to make sure everything is working exactly as we expect it to. To that end I wrote a lot of Rails system tests.

A system test is special because it does more than just test a method is run or what the output is. A system test is one where we actually run a web server, open a browser, and interact with the application testing along the way. This is pretty easy to get set up locally as Rails comes with basically everything it needs to run system tests out of the box coupled with what it assumes is on your computer but getting it to run in a test environment, like GitHub actions, can be a bit trickier. This has been a problem I've encountered in many projects and today I'm happy to share my configuration to get system tests running in GitHub actions.

Paylooza uses rspec as its testing suite but the modifications needed for minitest or any other suite won't be too different. Here are the important files and a bit of background about what they are and how they work.

Selenium Chrome Headless

Running your tests headlessly (i.e. without a Chrome browser window actually popping up and taking up your screen) can be a real benefit to your productivity locally and lends itself nicely when running the suite in a CI environment. Here is my rspec support file that I made to get the settings just right for headless system testing.

# spec/support/headless.rb
# frozen_string_literal: true

Capybara.register_driver :selenium_chrome_headless do |app|
  version = Capybara::Selenium::Driver.load_selenium
  options_key = Capybara::Selenium::Driver::CAPS_VERSION.satisfied_by?(version) ? :capabilities : :options
  browser_options = ::Selenium::WebDriver::Chrome::Options.new.tap do |opts|
    opts.add_argument("--headless")
    opts.add_argument("--disable-gpu") if Gem.win_platform?
    # Workaround https://bugs.chromium.org/p/chromedriver/issues/detail?id=2650&q=load&sort=-id&colspec=ID%20Status%20Pri%20Owner%20Summary
    opts.add_argument("--disable-site-isolation-trials")
    opts.add_preference("download.default_directory", Capybara.save_path)
    opts.add_preference(:download, default_directory: Capybara.save_path)
  end

  Capybara::Selenium::Driver.new(app, **{ browser: :chrome, options_key => browser_options })
end

RSpec.configure do |config|
  config.before(:each, type: :system) do
    driven_by :selenium_chrome_headless
  end
end

GitHub Action

Next, the GitHub action. GitHub actions can be very powerful and do a lot of different things and if you haven't read up on them then now's a pretty good time. In our use-case here we will be using them to run our test suite anytime a push is made to the project. This can be a push to a new branch before merging into main or a push into main from a branch or directly to ensure tests are passing before we deploy.

# .github/workflows/rspec.yml
name: tests
on: push

jobs:
  build:
    name: rspec
    runs-on: ubuntu-latest

    services:
      postgres:
        image: postgres
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
      redis:
        image: redis
        ports:
          - 6379:6379

    steps:
      - uses: actions/checkout@v2
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: 3.0.3
      - name: Install PostgreSQL
        run: sudo apt-get -yqq install libpq-dev
      - name: Run bundle install
        run: |
          gem install bundler
          bundle install --jobs 4 --retry 3
      - name: Run yarn commands
        run: |
          yarn install
          yarn build
          yarn build:css
      - name: Setup Database
        env:
          RAILS_ENV: test
          POSTGRES_HOST: localhost
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
        run: bin/rails db:create db:schema:load

      - name: Build and test with rspec
        env:
          RAILS_ENV: test
          POSTGRES_HOST: localhost
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
          RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
        run: bundle exec rspec

OK, that's a pretty big file so let's break it down piece by piece. The first few lines are all just normal GitHub actions config so I won't go into that. Let's look at the job itself.

...
    name: rspec
    runs-on: ubuntu-latest
...

Here we give the job a name of rspec and have it run on ubuntu-latest. Ubuntu is not the smallest distro we could use for this but everything plays pretty nicely with Ubuntu so it's pretty safe and easy to get working.

...
services:
      postgres:
        image: postgres
        env:
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
        ports:
          - 5432:5432
        options: >-
          --health-cmd pg_isready
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5
      redis:
        image: redis
        ports:
          - 6379:6379
...

Next we have services. These are side processes that we need running in order for our web server to run properly. This includes our postgres database and redis for background jobs and caching. It's important that we try and mimic our production environment as closely as we can during these tests so there are as few production-only bugs as possible.

...
 steps:
      - uses: actions/checkout@v2
      - uses: ruby/setup-ruby@v1
        with:
          ruby-version: 3.0.3
      - name: Install PostgreSQL
        run: sudo apt-get -yqq install libpq-dev
      - name: Run bundle install
        run: |
          gem install bundler
          bundle install --jobs 4 --retry 3
      - name: Run yarn commands
        run: |
          yarn install
          yarn build
          yarn build:css
      - name: Setup Database
        env:
          RAILS_ENV: test
          POSTGRES_HOST: localhost
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
        run: bin/rails db:create db:schema:load
...

Once our processes are up and running we can start to set up the environment for rspec to run. We'll need to checkout the code from GitHub, install necessary libraries like the one to communicate with Postgres, install Ruby libraries with bundle, install NPM packages with yarn, build our JavaScript and CSS files using yarn commands, and finally initialize our test database.

The yarn build and yarn build:css commands come from Rails 7 and may not be necessary for older builds of Rails. I was seeing that the application couldn't find the files unless they were built during set up.

...
- name: Build and test with rspec
        env:
          RAILS_ENV: test
          POSTGRES_HOST: localhost
          POSTGRES_USER: postgres
          POSTGRES_PASSWORD: postgres
          POSTGRES_DB: postgres
          RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
        run: bundle exec rspec

Finally we run bundle exec rspec with a few environment variables sprinkled in to make sure it connects properly. The most important one to note is RAILS_MASTER_KEY. This is used to decrypt the credentials file which has keys in it to communicate with services like Stripe and AWS. In GitHub you can save these secrets on your project so they can be accessed securely during actions by going to your project's Settings and then Secrets and adding an Action Secret in there.

Conclusion

That's it! Now I can sleep better knowing that nothing is going to go up to production without first going through our test suite. Now I just have to worry about the bugs I don't know exist yet. This took quite a lot of trial and error so I hope this is valuable to you. I'm sure there is some stuff that could be cleaned up but having this working is something I'm really happy with.