Joep Schuurkes (Posts about quality engineering)

Not continuous everything!

Joep Schuurkes — Sat, 20 Dec 2025 23:00:00 GMT

In Chapter 18 of "Taking Testing Seriously" (I skipped ahead, sue me), Keith Klain says:

So much today is about "continuous everything" - continuous deployment, continuous integration - and speed. The pace at which people are doing things is really, really disruptive to thinking deeply about problems. Then the goal gets displaced from getting good products and services to customers quickly, and turns into, "How do we back something out quickly when we inevitably screw up?" It's continuous everything - except continuous thinking.
- Taking Testing Seriously, James Bach, Michael Bolton et al., 2025, p. 424

He has a point. But I got there by first kind of disagreeing with what he said. Because "continuous everything" is not about speed. It's about risk. Or rather, about managing certain risks: the risk of (code) integration and the risks of delivery/deployment. And how fast you go, should not just be determined by those two specific risks. (Or even worse, by the fact that you just think it's really cool that you can release every commit to your users.) It should be determined by your overall risk appetite, which is a much broader perspective.

Some of the things I did after being off for a few weeks

Joep Schuurkes — Tue, 01 Jul 2025 22:00:00 GMT

I wish people would share more of what they do day-to-day. Especially testers and quality engineers. Show the work you do. On the other hand, I do realize that that's hard. You don't want to share sensitive information. You want to respect the privacy of your team. And then there's the whole practical side. How much of the context do you need to share to have an intelligible story?

Since my current project is open source and my team works in public, I have more options than most. So let's see if this kind of post works out.

Last week was my first week back at work after a few weeks of vacation. So an important part of my week was catching up. Going through emails and chat messages. Talk with my fellow team members. Participate in the usual meetings. Some stuff that didn't make it into this post. More importantly, there are three things I did, that give a good idea of what a significant part of my job looks like.

How essential are your testers?

Joep Schuurkes — Tue, 20 May 2025 22:00:00 GMT

A question I don't seem to be able to let go of is: where does a tester fit into a software development team? More specifically, how essential are testers to the team?

I think that there are basically three options:

Your testers do not add something essential.
It's not clear if your testers add something essential.
Your testers do add something essential.

Your testers do not add something essential

I suspect this is becoming more and more common. Thanks to Agile, CI/CD, DevOps, improved tooling and changing expectations, you should not have testers that find the dumb bugs I was expected to find at the start of my career. If a programmer wrote a > where it should have been a >=, we should expect a unit test and not a tester to catch it.

Optimizing for moments of discovery

Joep Schuurkes — Sat, 15 Feb 2025 23:00:00 GMT

This is the third post in a three-part series about my statement "The difference between a test case and a requirement is the moment of discovery."

Last year I wrote about how the difference between a requirement and a test case is the moment of discovery. And how that means that we should be intentional about our exploratory testing. Exploratory testing is just one example of a bigger idea, though: optimizing for moments of discovery.

So what does that mean, optimizing for moments of discovery? Don't those moments just happen? Isn't that what serendipity is all about? I think it's fair to say that you can't make moments of discovery happen. You can make them more likely to happen. That you can optimize for.

Before I go into two practices to optimize for these moments of discovery, I want to talk more generally about moving the moment of discovery, either earlier or later, for both requirements and test cases, i.e. for design and test. Because the two practices will do exactly that: moving the moment of discovery.

The nine skills of exploratory testing

Joep Schuurkes — Sat, 14 Dec 2024 23:00:00 GMT

Exploratory testing is a learned skill, as I claimed in my previous post "Being intentional about exploratory testing". In that post I mentioned the importance of two skills: noticing what there is to notice and deciding what to do next. Turns out it's not the first time I mentioned that pair of skills. In a post about how to teach Agile, I quoted John Mason's "Researching Your Own Practice, The Discipline of Noticing":

"All professional development could be described as changes in sensitivity to notice and accumulation of alternative actions to initiate." (p. 147)

That does raise the question if the skills of exploratory testing can't be made a little more specific. After giving it some thought, I came up with seven additional skills, making a total of nine. For some reasons they ended up as questions rather than nouns. I like how that makes this post less of a checklist and more of a tool for self-reflection. Each skill could be its own blog post, so I'm going to focus on one key element of each skill.

Being intentional about exploratory testing

Joep Schuurkes — Fri, 08 Nov 2024 23:00:00 GMT

This is the second post in a (to be) three-part series about my statement "The difference between a test case and a requirement is the moment of discovery."

In the previous post I distinguished test cases that are translated requirements from ones that aren't. This is something I learned from James Lyndsay. As he describes in "Why Exploration has a Place in any Strategy":

Some tests are designed to find risks. They're made on-the-fly and run once. Some are designed to tell us about retained value. They're made once, and run forever after. You need both: they tell you different things.

The tests with a focus on value are based on requirements, on things we know we want, they are prescribed (as in: written before). The tests with a focus on risks are exploratory, they are based on our decisions in the moment, we look for surprises and decide how we feel about those surprises.

One thing I've noticed through the years, is that a lot more exploratory testing is happening than we give credit for. It's hidden, a required but implicit part of the work. We do it, but we're not intentional about it.

Today I want to argue that it pays to be more intentional about exploratory testing. Before I get there, however, I want to explain what exploratory testing is, because there are still plenty of misconceptions going around.

What do you fix when you fix a test?

Joep Schuurkes — Sat, 24 Aug 2024 22:00:00 GMT

You ran the tests¹ - or a pipeline did it for you - and some of them failed. Time to fix the tests! But what is it exactly that needs fixing?

There are quite a few things that might make a test fail:

an issue with the build
an issue with the pipeline (if that's where the test runs)
an issue in the environment the code under test is running on
an issue in the environment the test code is running on
a bug in the code under test
a mistake in the test code
a mistake in what the test should test

Arguably, on the last three describe a test that fails. The test did its job detecting a problem. In the first four we didn't even get that far. The issues prevented the test from doing its job. So in those cases, it's not the test(s) as such that need fixing.

The difference between a test case and a requirement is the moment of discovery

Joep Schuurkes — Sun, 26 May 2024 22:00:00 GMT

There are several straightforward ways to distinguish a test case from a requirement. A test case tells you how to check some kind of thing about the application, a requirement tells you that the application should do some kind of thing. A test case is written by a tester, a requirement by a business analyst. A test case takes the shape of an action and an evaluation of the result, a requirement takes the form of a sentence like "product ABC shall do XYZ."¹

A less straightforward, but more interesting way to distinguish a test case and a requirement, is this:

The difference between a test case and a requirement is the moment of discovery.²

In this post I want to explore the meaning of that statement. In the next post I'll explore how looking at requirements and test cases in this way, can help us to do better testing. So this post will be a bit more philosophical, the next one more practical.

So you want to become a test engineer?

Joep Schuurkes — Sun, 25 Feb 2024 23:00:00 GMT

Becoming a test engineer these days is probably harder than it was for me back in 2006. Back then, there was no test automation, we worked in the slow rhythm of waterfall, and for years I was in a team with other testers or at least had a test manager to bounce ideas off. These days, there's a good chance none of these are true as you start as a test engineer.

While most of these changes are good ones (please don't take test automation or agile away), it does make me empathize with anyone who starts their career as a test engineer today. The pace is higher and the skill set is broader. More importantly, you need to navigate your career while no one is really sure where to position testers in their organization. That's not a straightforward environment to start a career in.

So here are four pieces of advice I'd give myself if I'd start my career in testing today:

testing can be many different things
you're a software engineer that specializes in testing
the end-game is leadership skills
shape your career in a way that suits you

A good tester is all over the place

Joep Schuurkes — Sat, 25 Nov 2023 23:00:00 GMT

Over the past year, I've been thinking about how testing-related roles are still an unsolved problem in software development. We keep trying different permutations: shifting left and shifting right, being closer to the programmers while not too far from other testers, doing less testing ourselves so we can support others more, etc.

And still, to be effective in any of these permutations, you can't let yourself be limited by them. You need to work both inside and outside the existing structures. You have to "be all over the place", in a good way.

Testers do testing

Let's start with a straightforward statement: a tester tests. Then what is testing? I still like the definition "Testing is investigating in order to evaluate a product." The most obvious thing to investigate, to test, is the code that is being written. The best way to do this, in my opinion, is through the combination of exploratory testing and test automation, i.e. what Maaret Pyhäjärvi has named "contemporary exploratory testing". And to be clear, while the execution part tends to be the most visible, effective testing also needs good test strategy, design, and reporting.