How to split slow RSpec test files by test examples (by individual it)?
❗ RSpec requirement: You need
RSpec >= 3.3.0 in order to use this feature.
In order to split RSpec slow test files by test examples across parallel CI nodes you need to set environment variable:
It allows a single test file to be auto split by test examples between parallel jobs (parallel CI nodes).
Thanks to that your CI build speed can be faster. It works with Knapsack Pro Regular Mode and Queue Mode.
Why I see error: Don't know how to build task
If you will see error like:
Don't know how to build task 'knapsack_pro:rspec_test_example_detector' (See the list of available tasks with `rake --tasks`)
It probably means bundler can't find the rake task. You can try to remove the default prefix
bundle exec used by
knapsack_pro gem by setting
How split by slow RSpec test files works?
Let's see an example. You have 2 test files:
spec/features/dashboard_spec.rb it takes 7 minutes to run. spec/features/homepage_spec.rb it takes 3 minutes to run.
In total, those 2 test files take 10 minutes to execute.
You run the test suite on 2 parallel CI nodes. In a perfect scenario, we would like to run each parallel job for 5 minutes because 10 minutes / 2 parallel jobs are 5 minutes. But we can't because one of the test files is slow. File
spec/features/dashboard_spec.rb takes 7 minutes to execute which is more than expected 5 minutes per parallel job. Because of this problem, your CI build will take 7 minutes to run instead of 5 minutes.
To fix the above problem we can use knapsack_pro auto split slow RSpec test files by test examples feature.
knapsack_pro knows that ideal expected time per parallel CI node is 5 minutes so it will look for test files that take more than
70% * 5 minutes = 3.5 minutes to run. It means if the test file takes
>= 3.5 minutes then it will be split by test examples between parallel CI nodes. We use the 70% threshold to adjust to randomness in the execution time of test files. The slower the test file the more probable is that it will run not always the same amount of time. This is especially happening for feature/capybara specs or when CI provider has some performance hiccup.
Is the test file split only by
RSpec test file is split by test examples. In RSpec syntax test example can be marked as
Why knapsack_pro does not split all test files by test examples?
Running test files by test examples add additional overhead for memory usage in RSpec. If you have a very large test suite (>= 1000 test files) then running all test files by test examples would lead to running out of RAM on your CI provider. There is no point in a split of all test files by test examples. Just a split of only bottleneck test files (all slow test files) is a good enough solution to make sure your CI build is as fast as possible.
How to auto split slow test files for other test runners
See other examples how to auto split test files by test cases on parallel jobs (CI nodes).