From 48299a7c0cfbbb41df7077f740b318433d3517eb Mon Sep 17 00:00:00 2001 From: jbranchaud Date: Wed, 3 Nov 2021 11:52:56 -0500 Subject: [PATCH] Add Find Minimal Set Of Tests Causing A Flicker as an RSpec TIL --- README.md | 3 ++- ...-minimal-set-of-tests-causing-a-flicker.md | 27 +++++++++++++++++++ 2 files changed, 29 insertions(+), 1 deletion(-) create mode 100644 rspec/find-minimal-set-of-tests-causing-a-flicker.md diff --git a/README.md b/README.md index ee98a76..9643a38 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ pairing with smart people at Hashrocket. For a steady stream of TILs, [sign up for my newsletter](https://crafty-builder-6996.ck.page/e169c61186). -_1162 TILs and counting..._ +_1163 TILs and counting..._ --- @@ -876,6 +876,7 @@ _1162 TILs and counting..._ ### RSpec - [Check Specific Arguments To Received Method](rspec/check-specific-arguments-to-received-method.md) +- [Find Minimal Set Of Tests Causing A Flicker](rspec/find-minimal-set-of-tests-causing-a-flicker.md) ### Ruby diff --git a/rspec/find-minimal-set-of-tests-causing-a-flicker.md b/rspec/find-minimal-set-of-tests-causing-a-flicker.md new file mode 100644 index 0000000..8387a57 --- /dev/null +++ b/rspec/find-minimal-set-of-tests-causing-a-flicker.md @@ -0,0 +1,27 @@ +# Find Minimal Set Of Tests Causing Flicker + +You have a pretty reliable test suite. However, every once in a while CI will +fail. It fails for some test that doesn't seem to be related to the PR. And if +you re-run CI, it may not fail a subsequent time. + +Your test suite has a flicker. + +Because this flickering test fails so sporadically, it can be hard to track +down when it fails and why. + +[RSpec comes with a `--bisect` +flag](https://relishapp.com/rspec/rspec-core/docs/command-line/bisect) that can +help you track down the _minimal_ sequence of tests that will produce a +failure. So, the next time CI fails unexpectedly on this flickering test, grab +the seed for that test run and use it locally to perform a bisect. + +```bash +$ rspec --seed 1234 --bisect +``` + +This while take a while to run, but when it is done, it should output an +`rspec` command with a series of specific tests. Copy, paste, and run this +command as you work on tracking down the issue. One strong possibility is that +one test is altering some global state in a way that the other test doesn't +expect. And it is only in this order that you can see that manifest as a +failure.