Joep Schuurkes (Posts about exploratory testing)

The nine skills of exploratory testing

Joep Schuurkes — Sat, 14 Dec 2024 23:00:00 GMT

Exploratory testing is a learned skill, as I claimed in my previous post "Being intentional about exploratory testing". In that post I mentioned the importance of two skills: noticing what there is to notice and deciding what to do next. Turns out it's not the first time I mentioned that pair of skills. In a post about how to teach Agile, I quoted John Mason's "Researching Your Own Practice, The Discipline of Noticing":

"All professional development could be described as changes in sensitivity to notice and accumulation of alternative actions to initiate." (p. 147)

That does raise the question if the skills of exploratory testing can't be made a little more specific. After giving it some thought, I came up with seven additional skills, making a total of nine. For some reasons they ended up as questions rather than nouns. I like how that makes this post less of a checklist and more of a tool for self-reflection. Each skill could be its own blog post, so I'm going to focus on one key element of each skill.

Being intentional about exploratory testing

Joep Schuurkes — Fri, 08 Nov 2024 23:00:00 GMT

This is the second post in a (to be) three-part series about my statement "The difference between a test case and a requirement is the moment of discovery."

In the previous post I distinguished test cases that are translated requirements from ones that aren't. This is something I learned from James Lyndsay. As he describes in "Why Exploration has a Place in any Strategy":

Some tests are designed to find risks. They're made on-the-fly and run once. Some are designed to tell us about retained value. They're made once, and run forever after. You need both: they tell you different things.

The tests with a focus on value are based on requirements, on things we know we want, they are prescribed (as in: written before). The tests with a focus on risks are exploratory, they are based on our decisions in the moment, we look for surprises and decide how we feel about those surprises.

One thing I've noticed through the years, is that a lot more exploratory testing is happening than we give credit for. It's hidden, a required but implicit part of the work. We do it, but we're not intentional about it.

Today I want to argue that it pays to be more intentional about exploratory testing. Before I get there, however, I want to explain what exploratory testing is, because there are still plenty of misconceptions going around.

Solving Black Box Puzzle 31 with data analysis

Joep Schuurkes — Sun, 28 Apr 2019 11:16:34 GMT

James Lyndsay has created a number of amazing Black Box Puzzles: tiny applications that challenge you to figure out what they do. (You can support him in creating more of these at his Patreon page.) Two of these Puzzles, 29 and 31, not only have a GUI to explore, but also an API.

And that gave me an idea. If you explore these Puzzles through their GUI, you start from the inputs. You try out different inputs in the hope of discovering a pattern in the outputs. And then that pattern feeds back into your exploration.
With an API, however - and because of the nature of Puzzle 31 - it becomes easy to get the outputs for all possible combinations of inputs. Which means you can start your exploration from the outputs instead of the inputs.

Before I tell you how and what I did, three important remarks.
First of all, I will be spoiling the solution to the Puzzle in this blog post. So this is the right moment to go and solve Puzzle 31 for yourself first. Or at least go play a bit with it, so you have an idea what the inputs and outputs are.
Secondly, I had already solved the Puzzle through the GUI a few months ago. So it was more of a "Can I find the solution this way as well?" than a "Can I find the solution?" thing.
Finally, the code and the spreadsheet I created (linked throughout, also available on GitHub here), are not very clean. I thought about tidying them up, but my two reasons for not doing so are (1) laziness; (2) the way they are now gives a more honest picture of what I did.

Some thoughts after attending the 'Getting a Grip on Exploratory Testing' workshop

Joep Schuurkes — Sun, 29 Apr 2012 17:55:30 GMT

About two weeks ago I attended James Lyndsay's 'Getting a Grip on Exploratory Testing' workshop in Amsterdam. So it's about time to write something about it…

Now one of the things I dislike about workshop blog posts is that people will say "It was great! And person X is such a good trainer!" without saying much about the content of the workshop. However, I find myself now writing this post and thinking: I shouldn't post a full summary of the workshop. Not that it would spoil too much for any future attendee: most of the workshop consists of exercises and discussion. But posting a summary of the workshop that James has put a lot of effort in to create, just doesn't feel right. So let me just say this: the workshop was great and James is such a good trainer! :-D

Now that's out of the way, there are a few things from the workshop I'd like to share. Of course, the usual disclaimer applies: these are my thoughts on what was presented during the workshop. Any misrepresentations are my responsibility.

The irony of scripted testing

Joep Schuurkes — Mon, 09 Apr 2012 21:16:17 GMT

A bit over a week ago, James Bach posted on twitter:

"This video shows a nice simple contrast between heavy scripted testing and exploratory testing http://youtu.be/PxTqjAwM2Pw"
- James Bach (@jamesmarcusbach) March 30, 2012

So I watched the video, hoping to see something that would make me go 'Cool!', but instead I went 'Hmmm.'

First let me say that this video does get a few things right:
- Exploratory testing can be structured by using charters.
- Exploratory testing allows you to easily change your test approach based on your test results.
- Exploratory testing is very adaptable when confronted with inaccurate or changing requirements.
Yet notice how the above only talks about exploratory testing, because for every thing the video gets right about exploratory testing, it gets something wrong about scripted testing – or rather about how scripted testing works in practice.