While working on Paylooza it was really important to me to have a very well tested application. Paylooza strives to be an affordable, off-the-shelf recurring billing solution for individuals and small businesses and because we are dealing with transactions it is important to make sure everything is working exactly as we expect it to. To that end I wrote a lot of Rails system tests.
A system test is special because it does more than just test a method is run or what the output is. A system test is one where we actually run a web server, open a browser, and interact with the application testing along the way. This is pretty easy to get set up locally as Rails comes with basically everything it needs to run system tests out of the box coupled with what it assumes is on your computer but getting it to run in a test environment, like GitHub actions, can be a bit trickier. This has been a problem I've encountered in many projects and today I'm happy to share my configuration to get system tests running in GitHub actions.
Paylooza uses rspec as its testing suite but the modifications needed for minitest or any other suite won't be too different. Here are the important files and a bit of background about what they are and how they work.
Selenium Chrome Headless
Running your tests headlessly (i.e. without a Chrome browser window actually popping up and taking up your screen) can be a real benefit to your productivity locally and lends itself nicely when running the suite in a CI environment. Here is my rspec support file that I made to get the settings just right for headless system testing.
Capybara.register_driver :selenium_chrome_headless do |app|
version = Capybara::Selenium::Driver.load_selenium
options_key = Capybara::Selenium::Driver::CAPS_VERSION.satisfied_by?(version) ? :capabilities : :options
browser_options = ::Selenium::WebDriver::Chrome::Options.new.tap do |opts|
opts.add_argument("--headless")
opts.add_argument("--disable-gpu") if Gem.win_platform?
opts.add_argument("--disable-site-isolation-trials")
opts.add_preference("download.default_directory", Capybara.save_path)
opts.add_preference(:download, default_directory: Capybara.save_path)
end
Capybara::Selenium::Driver.new(app, **{ browser: :chrome, options_key => browser_options })
end
RSpec.configure do |config|
config.before(:each, type: :system) do
driven_by :selenium_chrome_headless
end
end
GitHub Action
Next, the GitHub action. GitHub actions can be very powerful and do a lot of different things and if you haven't read up on them then now's a pretty good time. In our use-case here we will be using them to run our test suite anytime a push is made to the project. This can be a push to a new branch before merging into main or a push into main from a branch or directly to ensure tests are passing before we deploy.
name: tests
on: push
jobs:
build:
name: rspec
runs-on: ubuntu-latest
services:
postgres:
image: postgres
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
redis:
image: redis
ports:
- 6379:6379
steps:
- uses: actions/checkout@v2
- uses: ruby/setup-ruby@v1
with:
ruby-version: 3.0.3
- name: Install PostgreSQL
run: sudo apt-get -yqq install libpq-dev
- name: Run bundle install
run: |
gem install bundler
bundle install --jobs 4 --retry 3
- name: Run yarn commands
run: |
yarn install
yarn build
yarn build:css
- name: Setup Database
env:
RAILS_ENV: test
POSTGRES_HOST: localhost
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
run: bin/rails db:create db:schema:load
- name: Build and test with rspec
env:
RAILS_ENV: test
POSTGRES_HOST: localhost
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
run: bundle exec rspec
OK, that's a pretty big file so let's break it down piece by piece. The first few lines are all just normal GitHub actions config so I won't go into that. Let's look at the job itself.
...
name: rspec
runs-on: ubuntu-latest
...
Here we give the job a name of rspec and have it run on ubuntu-latest. Ubuntu is not the smallest distro we could use for this but everything plays pretty nicely with Ubuntu so it's pretty safe and easy to get working.
...
services:
postgres:
image: postgres
env:
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
ports:
- 5432:5432
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
redis:
image: redis
ports:
- 6379:6379
...
Next we have services. These are side processes that we need running in order for our web server to run properly. This includes our postgres database and redis for background jobs and caching. It's important that we try and mimic our production environment as closely as we can during these tests so there are as few production-only bugs as possible.
...
steps:
- uses: actions/checkout@v2
- uses: ruby/setup-ruby@v1
with:
ruby-version: 3.0.3
- name: Install PostgreSQL
run: sudo apt-get -yqq install libpq-dev
- name: Run bundle install
run: |
gem install bundler
bundle install --jobs 4 --retry 3
- name: Run yarn commands
run: |
yarn install
yarn build
yarn build:css
- name: Setup Database
env:
RAILS_ENV: test
POSTGRES_HOST: localhost
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
run: bin/rails db:create db:schema:load
...
Once our processes are up and running we can start to set up the environment for rspec to run. We'll need to checkout the code from GitHub, install necessary libraries like the one to communicate with Postgres, install Ruby libraries with bundle, install NPM packages with yarn, build our JavaScript and CSS files using yarn commands, and finally initialize our test database.
The yarn build and yarn build:css commands come from Rails 7 and may not be necessary for older builds of Rails. I was seeing that the application couldn't find the files unless they were built during set up.
...
- name: Build and test with rspec
env:
RAILS_ENV: test
POSTGRES_HOST: localhost
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
POSTGRES_DB: postgres
RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
run: bundle exec rspec
Finally we run bundle exec rspec with a few environment variables sprinkled in to make sure it connects properly. The most important one to note is RAILS_MASTER_KEY. This is used to decrypt the credentials file which has keys in it to communicate with services like Stripe and AWS. In GitHub you can save these secrets on your project so they can be accessed securely during actions by going to your project's Settings and then Secrets and adding an Action Secret in there.
Conclusion
That's it! Now I can sleep better knowing that nothing is going to go up to production without first going through our test suite. Now I just have to worry about the bugs I don't know exist yet. This took quite a lot of trial and error so I hope this is valuable to you. I'm sure there is some stuff that could be cleaned up but having this working is something I'm really happy with.
