🔍 ExUnit 1.17's new flag to find intermittent failures

Notes

Intermittent tests are a pain, but Elixir 1.17 has a really cool new flag in mix test.

Whenever we have a test that fails intermittently, we can run a set of tests N times until the test fails:

mix test test/path/to/failing/test.exs --repeat-until-failure 100

Once a test fails, you can grab the seed and run the failing test with that seed:

mix test test/path/to/failing/test.exs --seed 123456

If you’re curious as to how --seed affects our tests, take a look at ExUnit’s –seed surprising behavior