How many API tokens/keys do I need?
Each knapsack_pro command defined in the CI pipeline should use an individual API token. The purpose of an API token is to track a set of tests. When you run 2 different sets of tests within the same test suite (e.g., unit tests and feature tests are run separately), you need two separate API tokens.
Here are examples:
- Step 1.
bundle exec rake knapsack_pro:cucumber
- Step 2.
bundle exec rake knapsack_pro:rspec
- Step 3.
KNAPSACK_PRO_TEST_FILE_PATTERN="spec/features/*_spec.rb" bundle exec rake knapsack_pro:rspec
- Step 4.
bundle exec rake knapsack_pro:rspec[--tag tagA]
- Step 5.
bundle exec rake knapsack_pro:rspec[--tag ~tagA]
- Step 6.
bundle exec rake knapsack_pro:queue:rspec
- If you specified the
KNAPSACK_PRO_TEST_FILE_PATTERNenvironment variable to run a subset of the test suite, then you need a separate API token. This is useful when you want to track the same test file pattern across your branches. For instance, you want to run only unit tests without feature tests.
- If you pass
--tag ~tagA, then you run a subset of the test suite, so you need a separate API token.
- If you want to use Regular Mode and Queue Mode at the same time, then you need a separate API token for each mode.
When you run the same test suite for the different environments then you need multiple API tokens.
Let's say you have a single RSpec test suite and would like to run it concurrently against different environments like mobile and desktop. You should generate separate API tokens:
RSpec Mobile and
When you run the same test suite in steps on the CI server then you need multiple API tokens.
For instance, you run only unit tests and feature tests as a second step. Therefore, for each unique test file pattern defined with the
KNAPSACK_PRO_TEST_FILE_PATTERN environment variable, you need a separate API token.
This allows you to run both steps sequentially or in parallel as part of a CI build with the same build ID. Typically you would configure sequential steps in your CI provider, but some CI providers allow you to start parallel steps so that you can run unit tests and feature tests at the same time.
When should I use the same API token?
There is a scenario where you should use the same API token even though you are using a different set of tests.
Let's say you run all your RSpec tests on the
main branch. But on your feature branches, you want to run only a subset of the test suite that is related to the feature code changes. In such a case, you should use the same API token. This way, you can see all your branches in the Knapsack Pro dashboard on the list of CI builds for a given API token.
Run all tests on the
mainbranch and define a test file pattern with the
KNAPSACK_PRO_TEST_FILE_PATTERNenvironment variable for your feature branches to run a subset of the test suite.
Some users use the
crystallballgem. Its primary purpose is to select a minimal subset of your test suite which should be run to ensure your changes didn't break anything. You could run an RSpec test suite that takes 10 hours on the
mainbranch, but on the feature branch you may use custom tools or the
crystalballgem to detect what part of the codebase was modified and then run only a small subset of the 10 hours test suite. You can define a small subset of the test suite with the
KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILEenvironment variable (learn more). In such a case, you should use the same API token even though a test files list that's defined in the
KNAPSACK_PRO_TEST_FILE_LIST_SOURCE_FILEenvironment variable can be different for each CI build on your feature branch. The same API token is recommended so that Knapsack Pro can leverage collected test execution data to help you balance tests in Queue Mode well.