If you’re a manager who has testers reporting to you (or if you are a manager who has a manager who etc.), at some point you’re going to ask the question: “How do I know they are doing a good job?”

Answering that question may come with a few significant challenges attached. There’s a good chance you do not have a background in software testing yourself. You probably have a tool that can produce some numbers about testing with a dashboard highly recommended by its vendor. You also have plenty of other things on your plate, so whatever the solution is, it should not take up too much of your time. So how to proceed?

Before we go into that, though: this post is about telling if your testers are doing a good job with regards to testing. Testers are also team members, colleagues, employees, etc. This blog post is not about those things.

Read more…

In the episode “Thinking magically with Ramsey Dukes” of the “What magic is this?”-podcast, Ramsey Dukes (pen name of Lionel Snell) says some interesting things about thinking in twos as opposed to thinking in threes. (The quotes are from a section starting at 01:03:47 and ending at 01:08:02. Transcription mine.)

Thinking in twos and threes from the perspective of Magic

Thinking in twos, dualistic thinking, Ramsey Dukes calls a trick:

[…] the brain is constantly making distinctions. And, so I suggested that when we have these dualities, God - Devil is what we are sort of brought up with. […] The good, the bad. Now when that gets polarized. You see other people are always the bad. They’re the devil. And I say that if you look closely it’s actually a trick. All these things are tricks. […] There isn’t a little gate in the middle with a guard there, saying: “Fill in the form. Are you on this side or are you on that side?” It’s a whole spectrum. Everything’s connected.

Instead of thinking in twos, Ramsey Dukes recommends thinking in threes:

So I said that if instead, we trained ourselves or encouraged ourselves to think in threes. God, Devil, Trickster.

Where the third element of a duality is not the middle ground:

Read more…

A couple weeks ago I was updating some of our Playwright tests, when Playwright decided to mess with the execution order of the tests. The tests in question are a group of tests that cover a whole process from start to end. So we run them in serial mode, “to ensure they will always run together and in order”. And yet, they were not running in order.

While that’s obviously a bug, it wasn’t clear what exactly the bug was. Was it the tests not running in the correct order? Or was that only a symptom of some other bug? And what was triggering this bug? Because we’ve had these tests running in serial mode for a while, but had never seen this behavior before.

The problem as we encountered it

The tests that triggered the issue were similar to these:

Read more…

Recently I had to write a release advice. My first thought was: “I have close to two decades of experience in testing. I got this!” My second thought was: “I haven’t written a release advice in over ten years, how do I do this again?”

Luckily the first thought reflected reality a lot better than the second one. And after reviews by my team members, my project lead, and our manager, I’m proud of the end result. Even though in some ways the document has ended up more like a release decision, than a release advice. (More on that later.)

Lead with the actual advice

Initially I was going to start the document with a management summary, then an introduction, an overview of the testing that was done, the relevant known issues and remaining risks, then the actual release advice and finally some ideas on how to improve our testing further. Quite soon I realized that didn’t make sense.

Read more…

In Chapter 18 of “Taking Testing Seriously” (I skipped ahead, sue me), Keith Klain says:

So much today is about “continuous everything” - continuous deployment, continuous integration - and speed. The pace at which people are doing things is really, really disruptive to thinking deeply about problems. Then the goal gets displaced from getting good products and services to customers quickly, and turns into, “How do we back something out quickly when we inevitably screw up?” It’s continuous everything - except continuous thinking.
- Taking Testing Seriously, James Bach, Michael Bolton et al., 2025, p. 424

He has a point. But I got there by first kind of disagreeing with what he said. Because “continuous everything” is not about speed. It’s about risk. Or rather, about managing certain risks: the risk of (code) integration and the risks of delivery/deployment. And how fast you go, should not just be determined by those two specific risks. (Or even worse, by the fact that you just think it’s really cool that you can release every commit to your users.) It should be determined by your overall risk appetite, which is a much broader perspective.

Read more…

Testing types in software testing are a mess. It’s not just “regression testing” that can mean more things than there are people in the room. It’s true for basically all of them.

Just last week a teammate said “We should also do integration testing.” So I asked “What do you mean by integration testing?”. And the reply was: “Testing the frontend and backend together using Playwright.” Which made me think “I see what you mean, but I would never call that integration testing.” and made me reply: “Yeah, we should.”

The typical solution to this problem is to start defining the different testing types. To me, that’s just one approach of many to make sense of testing types. So far I’ve come up with an additional five.

Read more…

At the first conference I ever attended - or I think it was the first, in any case it was Let’s Test in 2012 -, I asked someone “So what kind of applications do you test?” And the reply was: “One of the onboard computers of a Saab fighter plane.” While at that time I was the test lead for an enterprise service bus.

We do not all have the same job.

In a recent episode of the Quality Remarks podcast hosted by Keith Klain, Paul Holland told the story of an organization where running the test automation took 13 people two weeks. And no, I’m sorry, they didn’t go into any detail about what happened in those two weeks. In my current project our pipeline takes 0 people about 17 minutes to run.

We do not all have the same job.

Read more…

In a previous post I claimed that any set of principles (or methodology for that matter) for good software development should cover these five areas:

• collaboration
• software engineering
• work management
• product
• reflect & experiment

I mentioned these five again in my post “What is a scrum master?”, but that’s all I’ve said about them. That changes with this post. I will provide a brief description for each of the five areas. I’ll share some ideas how a list of the five areas of Agile might be useful. And I’ll reflect on the question: Are they the five areas of good software development or of Agile software development?

Read more…