What do you fix when you fix a test?
You ran the tests1 - or a pipeline did it for you - and some of them failed. Time to fix the tests! But what is it exactly that needs fixing?
There are quite a few things that might make a test fail:
- an issue with the build
- an issue with the pipeline (if that’s where the test runs)
- an issue in the environment the code under test is running on
- an issue in the environment the test code is running on
- a bug in the code under test
- a mistake in the test code
- a mistake in what the test should test
Arguably, on the last three describe a test that fails. The test did its job detecting a problem. In the first four we didn’t even get that far. The issues prevented the test from doing its job. So in those cases, it’s not the test(s) as such that need fixing.