Joep Schuurkes (Posts about coverage)

What do you fix when you fix a test?

Joep Schuurkes — Sat, 24 Aug 2024 22:00:00 GMT

You ran the tests¹ - or a pipeline did it for you - and some of them failed. Time to fix the tests! But what is it exactly that needs fixing?

There are quite a few things that might make a test fail:

an issue with the build
an issue with the pipeline (if that's where the test runs)
an issue in the environment the code under test is running on
an issue in the environment the test code is running on
a bug in the code under test
a mistake in the test code
a mistake in what the test should test

Arguably, on the last three describe a test that fails. The test did its job detecting a problem. In the first four we didn't even get that far. The issues prevented the test from doing its job. So in those cases, it's not the test(s) as such that need fixing.

Three thoughts on risk-based testing

Joep Schuurkes — Sun, 17 Jul 2022 09:56:30 GMT

The past month I've been thinking about risk-based testing. This post collects three thoughts on risk-based testing I kept returning to.

If not based on risks, then based on what?

About a month ago I tweeted:

What are the alternatives to risk-based testing?
Requirements-based? I'd argue that's a subset of risk-based.
Unguided? That's either a bad idea ("we hope to get lucky") or aimed at the risk of unknown unknowns.
Any other options? Because something about the term is bothering me.

My question was inspired by the "opposite" heuristic from my talk about testing types: if we have some kind of category, what's the name for the things not in that category? Applied to risk-based testing: what's the name for testing that's not risk-based?

Measuring review coverage

Joep Schuurkes — Thu, 13 Sep 2012 17:39:57 GMT

Recently it occurred to me we have plenty of ways to measure test coverage, but there doesn't seem to be a way to measure review coverage. So today I decided to fix that. The result is the scale below with which you can measure review coverage. And just like all good scales, it goes all the way up to 11.

0) I'm sorry, what document?
1) I think I do remember someone mentioning that document.
2) I'm sure I have it somewhere – well, at least fairly sure...
3) Look, it's on my to do-pile!