Joep Schuurkes (Posts about philosophy of testing)https://smallsheds.garden/categories/cat_philosophy-of-testing.atom2023-11-26T10:50:52ZJoep SchuurkesNikolaWhy the testing/checking debate is so messy - a fruit salad analogyhttps://smallsheds.garden/blog/2015/why-the-testingchecking-debate-is-so-messy-a-fruit-salad-analogy/2015-11-15T12:17:12+01:002015-11-15T12:17:12+01:00Joep Schuurkes<div><p>Five days ago James Thomas posted <a href="https://www.linkedin.com/grp/post/55636-6069749695687770112?trk=groups-post-b-title">the following</a> in the Software Testing & Quality Assurance group on LinkedIn:</p>
<blockquote>
<p><strong>Are Testing and Checking different or not?</strong><br>
This <a href="http://gerrardconsulting.com/?q=node/659">article by Paul Gerrard</a> explains why we shouldn't be trying to draw a distinction between checking and testing, but should be paying more attention to the skills of the testers we employ to do the job.</p>
</blockquote>
<p>I posted a reply there, but I think I can do better than those initial thoughts, so here we go.</p>
<p>Let's imagine the following scene: Alice and Bob are preparing a fruit salad together.<br>
<em>Alice</em>: "Ok, let's make a nice fruit salad. We need some apples and some fruit."<br>
<em>Bob</em>: "Euh, aren't apples fruit?"<br>
<em>Alice</em>: "Yes. Of course. But when I say 'fruit', I mean 'non-apple fruit'."</p><p><a href="https://smallsheds.garden/blog/2015/why-the-testingchecking-debate-is-so-messy-a-fruit-salad-analogy/">Read more…</a> (3 min remaining to read)</p></div>What's the word for the part of testing that's not checking?https://smallsheds.garden/blog/2015/whats-the-word-for-the-part-of-testing-thats-not-checking/2015-08-17T20:19:25+02:002015-08-17T20:19:25+02:00Joep Schuurkes<div><h3>The question I asked</h3>
<p>Yesterday <a href="https://twitter.com/j19sch/status/632910141751447552">I asked on twitter</a>:</p>
<blockquote>
<p>Question: what's the proper word for the part of testing that's not checking? #cdt #testing #semantics<br>
- Joep Schuurkes (@j19sch) August 16, 2015</p>
</blockquote>
<p>The reason I asked, is that I noticed I needed that word in discussions about testing and checking. If checking is part of testing - and in the RST namespace it most definitely is, see '<a href="http://www.satisfice.com/blog/archives/856">Testing and checking refined</a>' -, then what can I contrast checking with? Contrasting checking with testing (as in 'checking versus testing') isn't going to work: there's one thing that's checking and then there's this other thing, testing, that contains that one thing and some other stuff<sup id="fnref:1"><a class="footnote-ref" href="https://smallsheds.garden/blog/2015/whats-the-word-for-the-part-of-testing-thats-not-checking/#fn:1">1</a></sup>, but it's like a completely different thing. See the difference? Conceptually that just doesn't work - at least not in my mind.</p>
<h3>The answers I got</h3>
<p>So I figured I'd ask twitter in all its infinite testing wisdom and lo and behold, not only did people reply, a discussion ensued with the following people (listed in no particular order) participating in different configurations: <a href="https://twitter.com/eddybruin">@eddybruin</a>, <a href="https://twitter.com/mariakedemo">@mariakedemo</a>, <a href="https://twitter.com/SandroIbig">@SandroIbig</a>, <a href="https://twitter.com/TestPappy">@TestPappy</a>, <a href="https://twitter.com/dwiersma">@dwiersma</a>, <a href="https://twitter.com/ilarihenrik">@ilarihenrik</a>, <a href="https://twitter.com/PhilipHoeben">@PhilipHoeben</a>, <a href="https://twitter.com/huibschoots">@huibschoots</a> and <a href="https://twitter.com/deefex">@deefex</a>. Thank you all!</p>
<p><a href="https://smallsheds.garden/blog/2015/whats-the-word-for-the-part-of-testing-thats-not-checking/">Read more…</a> (4 min remaining to read)</p></div>Three arguments against the verification-validation dichotomyhttps://smallsheds.garden/blog/2015/three-arguments-against-the-verification-validation-dichotomy/2015-03-24T20:53:24+01:002015-03-24T20:53:24+01:00Joep Schuurkes<div><p>Last week while talking with two colleagues, one of them mentioned the verification/validation thing. And I noticed it made me feel uneasy. Because I know well enough what is meant by the distinction, but on a practical level I simply can't relate to it. When I think about what I do as a software tester and how verification versus validation applies to it, nothing happens. Blank mind. Crickets. Tumbleweed.
So after giving it some thought, I present you with three arguments against the verification-validation dichotomy.</p>
<p>First of course, we have the obligatory interlude of defining these two terms. A place to start is the Wikipedia page on <a href="http://en.wikipedia.org/wiki/Software_verification_and_validation">Software verification and validation</a>. Unfortunately it contains conflicting definitions, so if anyone cares enough, please do fix. Luckily there's also the general <a href="http://en.wikipedia.org/wiki/Verification_and_validation">Verification and validation</a> page of Wikipedia, which gives us (among others) the tl;dr version of the distinction:</p>
<ul>
<li>Verification: <em>Are we building the product right?</em></li>
<li>Validation: <em>Are we building the right product?</em></li>
</ul>
<p>Finally there's the <a href="http://www.istqb.org/downloads/finish/20/145.html">ISTQB glossary v2.4</a> that borrows from ISO 9000:</p>
<ul>
<li>Verification: <em>Confirmation by examination and through provision of objective evidence that specified requirements have been fulfilled.</em></li>
<li>Validation: <em>Confirmation by examination and through provision of objective evidence that the requirements for a specific intended use or application have been fulfilled.</em></li>
</ul>
<p>Now on to the three arguments.</p>
<p><a href="https://smallsheds.garden/blog/2015/three-arguments-against-the-verification-validation-dichotomy/">Read more…</a> (3 min remaining to read)</p></div>The test case - an epistemological deconstructionhttps://smallsheds.garden/blog/2015/the-test-case-an-epistemological-deconstruction/2015-02-01T22:19:55+01:002015-02-01T22:19:55+01:00Joep Schuurkes<div><p><em>(This article was first published in Dutch in <a href="https://www.testnet.org/testnet/home">TestNet</a> Nieuws 18. The article below is a translation with minor changes. Many thanks to Joris Meerts and Ruud Cox for reviewing the original version.)</em></p>
<h3>Testing as an information problem</h3>
<p>Testing is an information problem. We are in search of certain information, of an answer to the question: does this application fulfill the relevant explicit and implicit expectations? The exact way in which we can answer this question, however, is not immediately clear. First we will need to decide which questions to ask, how to ask them and how to evaluate the responses. Hence, testing is an information problem.</p>
<p>For the traditional test methodologies (ISTQB and TMap being the most well-known) the test case is a large part of the solution. So let's take this solution apart epistemologically and see what it is we have in front of us. If the traditional test case is our solution, what information does a test case contain? What changes occur after executing it? And also, where is the understanding in all of this that's happening?</p>
<p>In this article, I will first describe how a typical test case is created and how it is used. Then we shall take a look at which kinds of information a test case contains. Finally, we will analyze where the understanding of what happens during testing, is present and where it is not.</p>
<p><a href="https://smallsheds.garden/blog/2015/the-test-case-an-epistemological-deconstruction/">Read more…</a> (10 min remaining to read)</p></div>Joining the fray on ISO 29119https://smallsheds.garden/blog/2014/joining-the-fray-on-iso-29119/2014-08-23T10:04:08+02:002014-08-23T10:04:08+02:00Joep Schuurkes<div><p>For those of you who weren't aware yet: something happened at CAST 2014 regarding the ISO/IEC/IEEE 29119 Software Testing. For instance, first <a href="http://www.youtube.com/watch?v=A721ltyVw3o&list=PLQB4l9iafcelXpJnK6IyDsoFeEb1icqrl">this</a> happened, which resulted in <a href="http://www.ipetitions.com/petition/stop29119">this</a> and <a href="http://www.professionaltestersmanifesto.org/">this</a> and a whole bunch of other initiatives.</p>
<p>And then someone on twitter (forgot who, apologies) suggested watching Stuard Reid's Eurostar webinar "ISO 29119 – the new set of international standards on software testing" which can be found <a href="https://www.youtube.com/watch?v=c4W-jWRldj0">here</a>. Since I liked to learn more about the standard, but not enough to pay almost 500€ to <a href="http://www.nen.nl/Zoekresultaten.htm?q=iso+29119">NEN</a> (Dutch standards organisation) for parts 1 to 3, I began watching the webinar.
And truth be told, I didn't watch all of it. Some parts were boring, some parts sounded quite reasonable and some parts I..euh..skipped. And that's ok because all I want to discuss is this one particular quote that begins at 33:25:</p>
<p><a href="https://smallsheds.garden/blog/2014/joining-the-fray-on-iso-29119/">Read more…</a> (2 min remaining to read)</p></div>Why I am context-drivenhttps://smallsheds.garden/blog/2014/why-i-am-context-driven/2014-06-30T07:06:21+02:002014-06-30T07:06:21+02:00Joep Schuurkes<div><p><em>(This post was first published on the DEWT site as part of a <a href="https://dewt.wordpress.com/category/why-am-i-context-driven/">blog post series</a> by the DEWT members .)</em></p>
<p>Why am I context-driven? Because it's more fun.</p>
<p>That's all there is to it.</p>
<p>Of course I could argue that becoming context-driven has made me a better tester and I do think it has. Yet it's not the reason I became a context-driven tester. Besides, how would I prove it made me a better tester?</p>
<p>So no, I am context-driven because it's more fun. Because it sees testing as an intellectual challenge. Because it allows human uncertainty to be at the core of what it is. Because it tells me that I'm in charge of what I do and how I do it. Because it encourages me to dive in at the deep end of some complex problem, trusting on my skills to get out on top and enjoying every step of the way.</p>
<p><a href="https://smallsheds.garden/blog/2014/why-i-am-context-driven/">Read more…</a> (1 min remaining to read)</p></div>DEWT2 - Becoming a context-driven testerhttps://smallsheds.garden/blog/2012/dewt2-becoming-a-context-driven-tester/2012-11-08T23:14:57+01:002012-11-08T23:14:57+01:00Joep Schuurkes<div><p>About a month ago ago (October 5th - 6th) I was in Driebergen to attend <a href="http://dewt.wordpress.com/2012/10/07/dewt2-was-a-blast/">DEWT2</a>, a peer workshop with as theme "Implementing Context-Driven Testing". As it turns out, implementing context-driven testing is not easy to do. That should not come as a surprise: it requires people to change and that is difficult. Luckily, I'm not a manager wanting to implement context-driven testing, so I can dodge most of that problem.</p>
<p>However, I do like 'spreading the word' on context-driven testing, because I would like for there to be more context-driven testers in the Netherlands (and Europe and the world, of course). So to promote context-driven testing I think there are three things I can do: <br>
1) set an example,<br>
2) be available to other people,<br>
3) leave bread crumbs.</p>
<p><a href="https://smallsheds.garden/blog/2012/dewt2-becoming-a-context-driven-tester/">Read more…</a> (4 min remaining to read)</p></div>VIPT - how to teach software testinghttps://smallsheds.garden/blog/2012/vipt-how-to-teach-software-testing/2012-07-29T16:24:51+02:002012-07-29T16:24:51+02:00Joep Schuurkes<div><p>In this final post on VIPT (Value-Information-Processes-Tools) it's time to take a look at teaching software testing. My previous posts on VIPT can be found <a href="https://smallsheds.garden/blog/2012/yet-another-testing-model-value-information-processes-value/">here</a>, <a href="https://smallsheds.garden/blog/2012/vipt-intermezzo-models-and-the-unix-philosophy/">here</a> and <a href="https://smallsheds.garden/blog/2012/vipt-bottom-up-or-top-down/">here</a>.</p>
<h3>A typical software testing course</h3>
<p>A typical traditional software testing course (at least in the way I have taught them) has three elements: theory, stories and exercises.</p>
<p>The first element is all about definitions (testing, test cases, defects, etc.), process descriptions and testing techniques (mostly test design). So basically what happens is that students get a brief introduction about testing in general and then we move on to the main part: teaching a specifc testing method.</p>
<p>The second element of the course are the stories. These are mostly stories aobut how testing in the real world does not work as described in the theory. At best they are stories containing all four elements of VIPT. Most of the time, however, they are just real-world examples of a certain definition or technique.</p>
<p>Finally, there are exercises. As with the techniques, these are mostly about test design. Unfortunately they are also very linear. There is only one correct answer and often only one correct way to get to that answer. So the main gist seems to be: "I taught you a trick, now show me you can perform the trick." But shouldn't learning about testing be more than learning to jump through a hoop on command?</p>
<p><a href="https://smallsheds.garden/blog/2012/vipt-how-to-teach-software-testing/">Read more…</a> (4 min remaining to read)</p></div>VIPT - bottom-up or top-downhttps://smallsheds.garden/blog/2012/vipt-bottom-up-or-top-down/2012-07-17T20:02:42+02:002012-07-17T20:02:42+02:00Joep Schuurkes<div><p>In this second post on VIPT I want to talk about bottom-up vs. top-down. The original plan for this post was to talk about the distance between tools and value, but in the past few days I figured out that bottom-up vs. top-down is a better approach.
If you don't know what VIPT is, please read <a href="https://smallsheds.garden/blog/2012/yet-another-testing-model-value-information-processes-value/">this previous post</a>. Don't worry, I'll wait.</p>
<h3>VIPT is a top-down model</h3>
<p>For me as a context-driven tester the VIPT model is very much a top-down thing. You analyze the context, find out what value you should/can deliver and then you proceed to information, processes and tools. Of course, that's easier said than done. Going through this for everything you do, requires a lot of time and effort. So most of the time you do quick analysis of the context, decide that it sufficiently resembles a context you encountered earlier and you use the same tools you used then. Most of the time that's ok - as long as you stay alert to any signs that you misread the context.<sup id="fnref:1"><a class="footnote-ref" href="https://smallsheds.garden/blog/2012/vipt-bottom-up-or-top-down/#fn:1">1</a></sup></p>
<p><a href="https://smallsheds.garden/blog/2012/vipt-bottom-up-or-top-down/">Read more…</a> (4 min remaining to read)</p></div>VIPT Intermezzo - Models and the Unix philosophyhttps://smallsheds.garden/blog/2012/vipt-intermezzo-models-and-the-unix-philosophy/2012-07-15T20:25:03+02:002012-07-15T20:25:03+02:00Joep Schuurkes<div><p>Thanks to <a href="https://testingcurve.wordpress.com/2012/07/09/yet-another-testing-model-value-information-processes-value/#comments">Neil Thompson's comments</a> on my previous post, I started thinking about what I want to do with the VIPT model. Do I want to expand and refine it to a grand unified theory of testing? And if not, then what?</p>
<h3>The Unix philosophy</h3>
<p>After some thinking I realized that with regard to models, I adhere to the <a href="http://www.faqs.org/docs/artu/ch01s06.html">Unix philosophy</a>.</p>
<h4>Do one thing and do it well</h4>
<p>Particularly I am thinking about the following quote from Doug McIlroy:</p>
<blockquote>
<p>"This is the Unix philosophy: Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface."</p>
</blockquote>
<p><a href="https://smallsheds.garden/blog/2012/vipt-intermezzo-models-and-the-unix-philosophy/">Read more…</a> (2 min remaining to read)</p></div>