Joep Schuurkes (Posts about testing and checking)

Why the testing/checking debate is so messy - a fruit salad analogy

Joep Schuurkes — Sun, 15 Nov 2015 11:17:12 GMT

Five days ago James Thomas posted the following in the Software Testing & Quality Assurance group on LinkedIn:

Are Testing and Checking different or not?
This article by Paul Gerrard explains why we shouldn't be trying to draw a distinction between checking and testing, but should be paying more attention to the skills of the testers we employ to do the job.

I posted a reply there, but I think I can do better than those initial thoughts, so here we go.

Let's imagine the following scene: Alice and Bob are preparing a fruit salad together.
Alice: "Ok, let's make a nice fruit salad. We need some apples and some fruit."
Bob: "Euh, aren't apples fruit?"
Alice: "Yes. Of course. But when I say 'fruit', I mean 'non-apple fruit'."

What's the word for the part of testing that's not checking?

Joep Schuurkes — Mon, 17 Aug 2015 18:19:25 GMT

The question I asked

Yesterday I asked on twitter:

Question: what's the proper word for the part of testing that's not checking? #cdt #testing #semantics
- Joep Schuurkes (@j19sch) August 16, 2015

The reason I asked, is that I noticed I needed that word in discussions about testing and checking. If checking is part of testing - and in the RST namespace it most definitely is, see 'Testing and checking refined' -, then what can I contrast checking with? Contrasting checking with testing (as in 'checking versus testing') isn't going to work: there's one thing that's checking and then there's this other thing, testing, that contains that one thing and some other stuff¹, but it's like a completely different thing. See the difference? Conceptually that just doesn't work - at least not in my mind.

The answers I got

So I figured I'd ask twitter in all its infinite testing wisdom and lo and behold, not only did people reply, a discussion ensued with the following people (listed in no particular order) participating in different configurations: @eddybruin, @mariakedemo, @SandroIbig, @TestPappy, @dwiersma, @ilarihenrik, @PhilipHoeben, @huibschoots and @deefex. Thank you all!

Test automation - five questions leading to five heuristics

Joep Schuurkes — Tue, 24 Mar 2015 19:53:24 GMT

I wrote a follow-up to this post in June 2019 in which I revisit the heuristics from this post.

In 1984 Abelson and Sussman said in the Preface to 'Structure and Interpretation of Computer Programs':

Our design of this introductory computer-science subject reflects two major concerns. First, we want to establish the idea that a computer language is not just a way of getting a computer to perform operations but rather that it is a novel formal medium for expressing ideas about methodology. Thus, programs must be written for people to read, and only incidentally for machines to execute. Second, we believe that the essential material to be addressed by a subject at this level is not the syntax of particular programming-language constructs, nor clever algorithms for computing particular functions efficiently, nor even the mathematical analysis of algorithms and the foundations of computing, but rather the techniques used to control the intellectual complexity of large software systems. [emphasis mine]

This oft-quoted sentence I emphasized, is even more true if the purpose of our programs is test automation¹. So let's say you run your test automation program and the result is a list of passes and fails. The purpose of testing is to produce information. You could say that this list of results qualifies as information and I would disagree. I would say it is data, data in need of interpretation. When we attempt this interpretation, we should consider the following five questions.