Thinking in threes about software testing

In the episode “Thinking magically with Ramsey Dukes” of the “What magic is this?”-podcast, Ramsey Dukes (pen name of Lionel Snell) says some interesting things about thinking in twos as opposed to thinking in threes. (The quotes are from a section starting at 01:03:47 and ending at 01:08:02. Transcription mine.)

Thinking in twos and threes from the perspective of Magic

Thinking in twos, dualistic thinking, Ramsey Dukes calls a trick:

[…] the brain is constantly making distinctions. And, so I suggested that when we have these dualities, God - Devil is what we are sort of brought up with. […] The good, the bad. Now when that gets polarized. You see other people are always the bad. They’re the devil. And I say that if you look closely it’s actually a trick. All these things are tricks. […] There isn’t a little gate in the middle with a guard there, saying: “Fill in the form. Are you on this side or are you on that side?” It’s a whole spectrum. Everything’s connected.

Instead of thinking in twos, Ramsey Dukes recommends thinking in threes:

So I said that if instead, we trained ourselves or encouraged ourselves to think in threes. God, Devil, Trickster.

Where the third element of a duality is not the middle ground:

One of the things people said to me: “I quite agree you know: there’s two extremes and there’s a sensible middle point.” I said: “No, no, no, no, no, no.” That sensible middle point is just saying this line is absolute. That third point lies out there.

Or put differently: a middle ground still affirms the duality. Things are still this or that, but by a matter of degree. While if you think in threes, the third thing is something that is actually different from the other two.

When you think in threes, the three elements are equal parts of a whole:

And these are three equal things: God, Devil, Trickster. […], we can play with those. And I say this isn’t something new. And I gave the example of alchemy.
Alchemy, ask the man in the street, “Oh that’s trying to turn lead into gold, isn’t it.” In other words, it’s dualistic thinking. There’s lead and there’s gold. Someone who knows a bit more and says: “Ah yes, earth, fire, air, and water. That’s alchemy isn’t it.” Four elements. […] Fewer people would say: “Ah yes, sulfur, salt, mercury.” Fewer people understand that.
It’s the same with astrology. Astrology people know there are masculine and feminine signs. And they know there are earth, fire, air and water signs. Fewer people know that you have cardinal, fixed, mutable.

And thinking in threes often results in change, in a flow:

When you have three things, I find there’s nearly always a flow. Sulfur, salt, mercury is the same as cardinal, fixed, mutable. Cardinal initiates, fixed sustains it, and then mutable: it’s time for it to converge in something else.

Finding a third principle to some dualities in software testing

There are two ways to apply thinking in threes to software testing. One is to find the third element to a commonly used duality. The other one is not to start from an existing duality, but to create something new. I started by extending some existing dualities and that helped me to come up with a new one. Which was a good thing, because the extended dualities I came up with, they were lacking something. As if they couldn’t entirely shake off their earlier existence as a duality.

Verification and validation

Verification answers the question “Are we building the thing right?” Validation answers the question “Are we building the right thing?”¹

Since both of these are confirmatory questions, a third element we could add is disconfirmation: “Can we make the thing do something wrong?” That doesn’t really result in change or a flow, though. At best, making it do something wrong, helps us realize a new aspect of “right”.

So as a third element to add to verification and validation, I would propose specification: “What is right?” And then we have somewhat of a flow, where specification leads to verification and validation, and both of those feed back into specification.

Manual and automated

Tests can be executed manually or they can be automated. Often the “automated” hides a lot of manual work in running the tests and evaluating the results. But running automated tests manually sounds like we didn’t do a good job of automating, so let’s not say the quiet part out loud.

Which brings us to the third element: idea. Because regardless of whether the tests are run manually or automated, where did these tests come from? Someone had to have the idea for a test and design it.

And the first time you run a test, it’s manually. Even a unit test: you write the test and you run it. The fact that you use code to interact with the software you’re testing instead of a mouse and keyboard, does not make it any less of a manual test.

And then we have a flow and even one that initiates, sustains and mutates: idea - manual - automated. What I like about this set of threes is that it pulls in test ideas into a duality that is purely about execution. What I dislike about it is that it suggest that any test idea can be automated. And that’s simply not true.

Maybe a better choice then for the third element is pipeline. It sticks to the original scope of the duality, i.e. execution, but highlights that “automated” and “automated” are not always the same thing. It also makes me uncomfortable, because it does not question the premise of the duality that testing is done either manually or automated. And that the more automated, the better.

Exploratory and scripted

Testing can be exploratory: you decide what happens next. Or it can be scripted: the script decides what will happen next.

Building on what I wrote above about manual and automated, the missing third element here is automation. Some form of a fully automated execution of a script.

All tests start as something exploratory. You have an idea for a test and you perform the test. Then it gets scripted. Some of your performance gets scripted into something that’s (largely) repeatable. Ideally that’s code, because then that code can be put in a pipeline. And that pipeline then forms a solid base to start exploring again.

In every step along the way, something is lost, though. The active human involvement gets smaller which each step. So in a pipeline you end up with something many times smaller than what you had while exploring. Which leads me to the fully new set of three I came up with.

A new set of threes: exploration - formalization - execution

This new set of threes is not based on an existing, common duality in software testing. There are definitely links to the three sets of threes I described above, though. Nothing is ever entirely new.

This set of threes is : exploration - formalization - execution.

First, you explore. You discover things about the thing you’re testing. You evaluate these discoveries. Are they good things? Bad things? Things that need more investigation?

Second, you formalize. You create a diagram, a list of test cases/ideas, some code, a summary of your testing session. Something that captures whatever needs to be preserved, so it can be used in the future.²

Third, you execute. You use what was formalized to do more testing. It’s not about blindly following what was transmitted to you, however. Even if the execution is no more than looking at the results of what was run automatically in a pipeline. You still need to be thinking and evaluating.

Last, but not least, the execution feeds back into exploration. You notice things that warrant further investigation. You realize something important was lost during formalization and execution. And so you return to exploration and the cycle continues.

---

Echoes of thinking in threes

According to Magic everything is connected. There are two ideas that I really like, that echo this thinking in threes.

Virginia Satir and having three choices

From page 143 of the book “The Satir Model”:

Considering three choices, three meanings, three responses, and three actions when looking at changing old patterns is very helpful for clients. Satir encouraged people to have at least three choices. She said to have one choice is no choice; to have two choices is a dilemma; and to have three choices offers new possibilities.

Thinking in twos creates an either/or. Thinking in threes creates freedom and movement.

Cynefin and aporia

Cynefin with its five domains is not purely static. There are three dynamics of liminal Cynefin. Each of the three is (part of) a loop, which echoes the initiate, sustain and mutate of thinking in threes. These loops move from the complex domain, to or towards the complicated domain, through aporia (aka the aporetic turn), potentially through the chaotic domain (a shallow dive), to return to the complex domain. So complex - complicated - aporia echoes initiate - sustain - mutate.

And just today I discovered Dave Snowden published a post titled “Trialectics, or thinking in threes” earlier this month:

Binary structures are not useless. They are often the right tool. But they share a fundamental limitation: they presuppose that the space of possibilities is adequately described by two positions and the tension between them. When that presupposition holds, they work. When it does not, binary thinking does not merely fail to capture what is happening. It actively prevents you from seeing it.

A trialectic is a three-term structure in which the terms are not positions on a spectrum, not options in a choice, and not thesis and antithesis awaiting synthesis. They are ontologically distinct modes, each appropriate to different conditions, each capable of causing real damage when applied in conditions that call for one of the others.

That is the structure a trialectic requires. Not three positions on a line, not three stages in a sequence, not three options in a choice, but three modes in permanent tension whose relationship is itself the object of attention.

---

Postscript: Thinking in fours

Ramsey Dukes also warns for thinking in fours, because it leads to stereotyping:

When you have a divide. North South. East West. You get a North-South divide. You get a East-West conflict. You don’t actually have a North South East West conflict, do you? But you get stereotyping. He’s a typical Northener. Oh Eastern mysticism. Oh Western arrogance. You stereotype when you have four things, you stereotype.

And in a video about his books:

If you think in fours, you get into stereotyping. Earth, air, water, fire: what type are you?

I suspect anyone who’s ever seen a quadrant (whether it was a magic one or not), can relate to this.