Demos & Benchmarks
See Brisk in action — real-world test suites running 10–40× faster across popular frameworks and CI/CD platforms.
CI/CD Integration
Running Brisk in Your CI/CD Pipeline
GitHub Actions
Run Brisk in your CI/CD pipeline
GitHub Actions is a CI/CD pipeline that is built into GitHub. It is free for public repositories and has a generous free tier for private repositories.
The following examples show how to run tests using Brisk from your GitHub Actions pipeline. The advantage of running Brisk from GitHub Actions is that you can take advantage of the massive speed improvement that Brisk provides. By running from GitHub Actions however, you can easily integrate with your existing code repo, pass and fail is just a GitHub Actions run which has great integration with GitHub. This allows you to quickly integrate Brisk into your existing workflow.
You can go to this live example to see a GitHub Actions run. We use the React codebase as an example project. It uses Jest. This example runs in 29 seconds on 20 workers.
CLI Tool
As a CLI From Your Terminal
Brisk was originally designed — and still works — as a CLI tool. You can use it to run all of your tests from your development environment in the cloud.
The following examples show how to run tests using Brisk from your terminal. The setup will also work with your CI/CD pipeline if you want to run tests automatically.
Jest — React Test Suite
React codebase running on Brisk
A side-by-side run of Brisk versus running Jest locally.
In this demo we use the React codebase (cloned from https://github.com/facebook/react ) as our source folder. Without Brisk it takes 3 minutes 12 seconds to run the test suite on my machine. With Brisk we can run it completely in 12 seconds. Furthermore, with Brisk this 12 seconds is the time of the slowest test. If we worked on optimizing that test we should see an even faster run. Also, adding additional tests shouldn't make our test runs slower, provided we have adequate concurrency and they themselves are under 12 seconds.
Jest is running in the top pane and the same test suite running through Brisk is on the bottom. With Brisk we can run the entire test suite before Jest has finished running a single test. I kick off a Jest run manually then switch panes to Brisk and hit "r" to initiate a run. All of the tests run through Brisk in the time it takes for Jest to load locally.
This demo was run on a 2020 MacBook Pro with 16 GB RAM.
brisk.json
{
"commands": [
{
"commandline": "yarn test --json "
}
],
"preSyncCommands": [],
"buildCommands": [
{ "commandline": "nvm install 14.18.0" },
{ "commandline": "nvm use 14.18.0" },
{ "commandline": "nvm alias default 14.18.0" },
{ "commandline": "yarn " }
],
"remoteDirectory": "/data/remote_dir/",
"environment": {},
"excludedFromSync": ["log/", ".git/", "node_modules"],
"excludedFromWatch": ["log/", ".git/", "log", ".git", "node_modules"],
"projectToken": "********",
"framework": "Jest"
}
Rails RSpec — Dev.to Codebase
Forem / Dev.to running on Brisk
The Dev.to website is released as an open source project called Forem. We are using it here as an example Rails with RSpec project.
It is a real world project that is actively maintained and has a large community of contributors. It is a great example of a large Rails project that has grown over the years. Like most real world Rails apps, functionality has increased and so has test size and speed as development has progressed. The test suite is extensive and includes browser tests using Capybara and Selenium.
Running the entire test suite locally takes a long time. I stopped at 60 minutes because my laptop was starting to overheat and was becoming unresponsive. This is pretty common for large Rails projects. Developers often cannot run their tests locally anymore — which is the problem Brisk is here to fix.
Brisk running the entire Forem / Dev.to test suite in 1 minute 30 seconds instead of > 60 minutes.
In the recording we run the entire test suite in 1 minute 30 seconds. This includes all of the RSpec tests including browser tests. This is at least a 40X speedup, from over 60 minutes to 1 minute 30 seconds and makes it feasible for developers to run their tests pre-commit again.
With Brisk CI, it means that test run times are no longer a bottleneck for CI. Merging and deploying code is faster and more reliable.
Making Things Even Faster
The big drag on test speed in this test suite is the browser tests. We spin up a Chrome instance with Selenium to test actual interactions with the browser. Unfortunately this can be quite flaky and so it is common to use rspec-retry to retry the tests if they hit timeouts. This doesn't solve the actual flakiness but does retry it. With such a large number of tests we do hit some retries which result in slowing down the entire test suite (because we are waiting on the slowest test which may have been retried a number of times). I would estimate that if some attention was paid to the flakiness of the browser tests we could get the test suite down to under 1 minute, perhaps less than 40 seconds. If we also split out the longer tests into multiple shorter tests with fewer test cases per file we could speed things up even further.
Some of the following demos are outdated and show older versions of Brisk — they are preserved here for reference until updated.
Rails Minitest — ActiveRecord Test Suite
Rails codebase running on Brisk
In this demo we use the Rails codebase (cloned from https://github.com/rails/rails) as our source folder. We are running a subset of the tests, the ones from the activerecord directory.
brisk.json
{
"listTestCommand": "./listTests.sh",
"commands": [
{
"commandline": "cd activerecord bin/test ",
"args": []
}
],
"preSyncCommands": [],
"buildCommands": [
{ "commandline": "bundle install", "args": [] }
],
"environment": {},
"excludedFromSync": ["log/", ".git/", "node_modules", ".rvm"],
"excludedFromWatch": ["log/", ".git/", "log", ".git", "node_modules"],
"projectToken": "********",
"framework": "Rails",
"image": "rails"
}
Example Usage
Below is a video of the demo showing running the tests using Brisk. This is a subset of the tests, running only the activerecord tests. Running the tests on my machine locally (a MacBook Pro 2020) takes over 2 minutes 25 seconds. Running in Brisk it completes in less than 19 seconds.
Making Things Even Faster
Running on Brisk each test run takes less than 20 seconds. At this point we are limited by the slowest test. It's actually trivial to increase parallelizability for most tests. Most people don't because there is normally no reason to, but now there is.
Cypress — Kitchensink
Cypress codebase running on Brisk
In this demo we use the Cypress Kitchensink codebase (cloned from cypress-example-kitchensink ) as our source folder.
Brisk running the Cypress Kitchensink test suite.
brisk.json
{
"listTestCommand": "/data/remote_dir/listTest.sh",
"commands": [
{
"commandline": "./node_modules/.bin/cypress run",
"workDirectory": "/data/remote_dir/",
"args": [],
"background": false
}
],
"preSyncCommands": [],
"buildCommands": [
{ "commandline": "nvm install 14.18.0", "workDirectory": "/data/remote_dir/", "args": [] },
{ "commandline": "nvm use 14.18.0", "workDirectory": "/data/remote_dir/", "args": [] },
{ "commandline": "nvm alias default 14.18.0", "workDirectory": "/data/remote_dir/", "args": [] },
{ "commandline": "npm ci", "workDirectory": "/data/remote_dir/", "args": [] },
{ "commandline": "npm run start:ci", "workDirectory": "/data/remote_dir/", "args": [], "background": true }
],
"environment": {},
"excludedFromSync": ["log/", ".git/", "node_modules"],
"excludedFromWatch": ["log/", ".git/", "log", ".git", "node_modules"],
"projectToken": "********",
"framework": "Cypress",
"image": "node-lts"
}
Running the tests on my machine locally (a MacBook Pro 2019) takes over 2 minutes 44 seconds. Running in Brisk it completes in less than 41 seconds.
Making Things Even Faster
Cypress is interesting because it has a lot of setup. We think we could make it faster, but this is also close to our max time — adding more tests won't make it slower, provided we add more workers.
Stop Waiting For Tests. Start your free trial today.
Join thousands of developers who have already accelerated their testing workflow with Brisk.