Joep Schuurkes (Posts about software testing)

If developers have autonomy, mastery and purpose, what do you need testers for?

Joep Schuurkes — Fri, 18 Sep 2026 22:00:00 GMT

At the latest FroGS conf open space, I hosted a session around the question:

If developers have autonomy, mastery and purpose, what do you need testers for?

Not because I believe that obviously in that case you don't, but I thought exploring the question would clarify the different roles a tester might have in different organizations.

After briefly sharing my own answers and my critique on the premise of the question, we had a great discussion with the group. As this post is based on the session as a whole (without claiming to cover everything that was discussed), I want to give a big thank you to everyone who participated in it.

Sidenote: for a post about a different session at the latest FroGS conf, read Elizabeth Zagroba's post "Tortured Fish Metaphors".

How do I know my testers are doing a good job?

Joep Schuurkes — Tue, 30 Jun 2026 22:00:00 GMT

If you're a manager who has testers reporting to you (or if you are a manager who has a manager who etc.), at some point you're going to ask the question: "How do I know they are doing a good job?"

Answering that question may come with a few significant challenges attached. There's a good chance you do not have a background in software testing yourself. You probably have a tool that can produce some numbers about testing with a dashboard highly recommended by its vendor. You also have plenty of other things on your plate, so whatever the solution is, it should not take up too much of your time. So how to proceed?

Before we go into that, though: this post is about telling if your testers are doing a good job with regards to testing. Testers are also team members, colleagues, employees, etc. This blog post is not about those things.

Thinking in threes about software testing

Joep Schuurkes — Sat, 30 May 2026 22:00:00 GMT

In the episode "Thinking magically with Ramsey Dukes" of the "What magic is this?"-podcast, Ramsey Dukes (pen name of Lionel Snell) says some interesting things about thinking in twos as opposed to thinking in threes. (The quotes are from a section starting at 01:03:47 and ending at 01:08:02. Transcription mine.)

Thinking in twos and threes from the perspective of Magic

Thinking in twos, dualistic thinking, Ramsey Dukes calls a trick:

[...] the brain is constantly making distinctions. And, so I suggested that when we have these dualities, God - Devil is what we are sort of brought up with. [...] The good, the bad. Now when that gets polarized. You see other people are always the bad. They're the devil. And I say that if you look closely it's actually a trick. All these things are tricks. [...] There isn't a little gate in the middle with a guard there, saying: "Fill in the form. Are you on this side or are you on that side?" It's a whole spectrum. Everything's connected.

Instead of thinking in twos, Ramsey Dukes recommends thinking in threes:

So I said that if instead, we trained ourselves or encouraged ourselves to think in threes. God, Devil, Trickster.

Where the third element of a duality is not the middle ground:

It's clearly a bug, but what is the bug exactly?

Joep Schuurkes — Sat, 07 Mar 2026 23:00:00 GMT

A couple weeks ago I was updating some of our Playwright tests, when Playwright decided to mess with the execution order of the tests. The tests in question are a group of tests that cover a whole process from start to end. So we run them in serial mode, "to ensure they will always run together and in order". And yet, they were not running in order.

While that's obviously a bug, it wasn't clear what exactly the bug was. Was it the tests not running in the correct order? Or was that only a symptom of some other bug? And what was triggering this bug? Because we've had these tests running in serial mode for a while, but had never seen this behavior before.

The problem as we encountered it

The tests that triggered the issue were similar to these:

Reflections after writing a release advice

Joep Schuurkes — Sat, 07 Feb 2026 23:00:00 GMT

Recently I had to write a release advice. My first thought was: "I have close to two decades of experience in testing. I got this!" My second thought was: "I haven't written a release advice in over ten years, how do I do this again?"

Luckily the first thought reflected reality a lot better than the second one. And after reviews by my team members, my project lead, and our manager, I'm proud of the end result. Even though in some ways the document has ended up more like a release decision, than a release advice. (More on that later.)

Lead with the actual advice

Initially I was going to start the document with a management summary, then an introduction, an overview of the testing that was done, the relevant known issues and remaining risks, then the actual release advice and finally some ideas on how to improve our testing further. Quite soon I realized that didn't make sense.

Testing types - beyond definitions

Joep Schuurkes — Mon, 08 Dec 2025 23:00:00 GMT

Testing types in software testing are a mess. It's not just "regression testing" that can mean more things than there are people in the room. It's true for basically all of them.

Just last week a teammate said "We should also do integration testing." So I asked "What do you mean by integration testing?". And the reply was: "Testing the frontend and backend together using Playwright." Which made me think "I see what you mean, but I would never call that integration testing." and made me reply: "Yeah, we should."

The typical solution to this problem is to start defining the different testing types. To me, that's just one approach of many to make sense of testing types. So far I've come up with an additional five.

We do not all have the same job

Joep Schuurkes — Sat, 15 Nov 2025 23:00:00 GMT

At the first conference I ever attended - or I think it was the first, in any case it was Let's Test in 2012 -, I asked someone "So what kind of applications do you test?" And the reply was: "One of the onboard computers of a Saab fighter plane." While at that time I was the test lead for an enterprise service bus.

We do not all have the same job.

In a recent episode of the Quality Remarks podcast hosted by Keith Klain, Paul Holland told the story of an organization where running the test automation took 13 people two weeks. And no, I'm sorry, they didn't go into any detail about what happened in those two weeks. In my current project our pipeline takes 0 people about 17 minutes to run.

We do not all have the same job.

Some of the things I did after being off for a few weeks

Joep Schuurkes — Tue, 01 Jul 2025 22:00:00 GMT

I wish people would share more of what they do day-to-day. Especially testers and quality engineers. Show the work you do. On the other hand, I do realize that that's hard. You don't want to share sensitive information. You want to respect the privacy of your team. And then there's the whole practical side. How much of the context do you need to share to have an intelligible story?

Since my current project is open source and my team works in public, I have more options than most. So let's see if this kind of post works out.

Last week was my first week back at work after a few weeks of vacation. So an important part of my week was catching up. Going through emails and chat messages. Talk with my fellow team members. Participate in the usual meetings. Some stuff that didn't make it into this post. More importantly, there are three things I did, that give a good idea of what a significant part of my job looks like.

How essential are your testers?

Joep Schuurkes — Tue, 20 May 2025 22:00:00 GMT

A question I don't seem to be able to let go of is: where does a tester fit into a software development team? More specifically, how essential are testers to the team?

I think that there are basically three options:

Your testers do not add something essential.
It's not clear if your testers add something essential.
Your testers do add something essential.

Your testers do not add something essential

I suspect this is becoming more and more common. Thanks to Agile, CI/CD, DevOps, improved tooling and changing expectations, you should not have testers that find the dumb bugs I was expected to find at the start of my career. If a programmer wrote a > where it should have been a >=, we should expect a unit test and not a tester to catch it.

Optimizing for moments of discovery

Joep Schuurkes — Sat, 15 Feb 2025 23:00:00 GMT

This is the third post in a three-part series about my statement "The difference between a test case and a requirement is the moment of discovery."

Last year I wrote about how the difference between a requirement and a test case is the moment of discovery. And how that means that we should be intentional about our exploratory testing. Exploratory testing is just one example of a bigger idea, though: optimizing for moments of discovery.

So what does that mean, optimizing for moments of discovery? Don't those moments just happen? Isn't that what serendipity is all about? I think it's fair to say that you can't make moments of discovery happen. You can make them more likely to happen. That you can optimize for.

Before I go into two practices to optimize for these moments of discovery, I want to talk more generally about moving the moment of discovery, either earlier or later, for both requirements and test cases, i.e. for design and test. Because the two practices will do exactly that: moving the moment of discovery.