testing
our testing types

Romanian Testing Conference logo
Joep Schuurkes - @j19sch
slides at j19sch.github.io
Mendix logo

testing type or test type?

Google Ngram test type vs testing type

testing our testing types

"experience in functional testing, whitebox testing, regression testing, mobile testing and UI testing"

"we should do integration testing"

is it really that difficult?

How Google tests software

"Instead of distinguishing between code, integration, and system testing, Google uses the language of small, medium, and large tests, emphasizing scope over form."

How Google Tests Software - Whittaker, Arbon, Carollo

testing our testing types

Wittgenstein: philosophy is an activity

definitions

Merriam-Webster's definition of definition

definitions

integration testing
Testing performed to expose defects in the interfaces and in the interactions between integrated components or systems.
regression testing
Testing of a previously tested component or system following modification to ensure that defects have not been introduced or have been uncovered in unchanged areas of the software, as a result of the changes made.
negative testing
Tests aimed at showing that a component or system does not work. Negative testing is related to the tester's attitude rather than a specific test approach or test design technique, e.g., testing with invalid input values or exceptions.
source: ISTQB Glossary

definitions all the way down

black-box testing
Testing, either functional or non-functional, without reference to the internal structure of the component or system.

definitions all the way down

black-box testing
Testing, either functional or non-functional, without reference to the internal structure of the component or system.
component
A minimal part of a system that can be tested in isolation.
system
A collection of interacting elements organized to accomplish a specific function or set of functions.

definitions all the way down

black-box testing
Testing, either functional or non-functional, without reference to the internal structure of the component or system.
(non-)functional testing
Testing conducted to evaluate the compliance of a component or system with (non-)functional requirements.
compliance
The capability of the software product to adhere to standards, conventions or regulations in laws and similar prescriptions.
functional requirement
A requirement that specifies a function that a component or system must be able to perform.
non-functional requirement
A requirement that describes how the component or system will do what it is intended to do.

definitions all the way down

black-box testing
Testing, either functional or non-functional, without reference to the internal structure of the component or system.
testing
The process consisting of all lifecycle activities, both static and dynamic, concerned with planning, preparation and evaluation of software products and related work products to determine that they satisfy specified requirements, to demonstrate that they are fit for purpose and to detect defects.

definitions

relations between concepts

assume a starting point of knowledge

opposites

Select all squares not a pipe

opposites

white-box vs black-box testing

functional vs non-functional testing

dynamic vs static testing

scripted vs exploratory testing

negative vs positive testing

terms without opposite

agile testing

developer testing

regression testing

opposites

explore meaning through what something isn't

what does it mean if there is no opposite?

danger of false dichotomies

Wittgenstein

Wittgenstein in 1922

Wittgenstein

26 April 1889: born in Vienna

1906 - 1911: studies engineering in Berlin and Manchester

1911: arrives in Cambridge to study philosophy with Bertrand Russell

1914-1918: fights for the Austrian army on the Russian and Italian front

1919: finishes Tractatus Logico-Philosophicus

1919: becomes a school teacher

1921: Tractatus Logico-Philosophicus published

Tractatus Logico-Philosophicus

sense - the empirical
propositions corresponding to a state of affairs in the world

senseless - the logical
tautologies and contradictions

nonsense - the transcendental
art, ethics, values, aesthetics, philosophy, meaning of life and religion

philosophy is solved!

all the elements

Periodic table of elements

all the elements

quality characteristic testing
performance, security, functional, accessibility, usability

device type testing (?)
desktop, mobile, tablet, browser, mainframe testing

all the elements

unit testing

unit - component - system - integration - end-to-end
unit - api - ui

test automation pyramid

here be pyramids - Joris Meerts
Test automation pyramid, Cohn 2009

all the elements

elements in a category clarify each other's meaning

in what way are they the same?
in what way are they the different?

combinations

food combinations

combinations

manual unit testing

manual API testing

automated usability testing

combinations

scripted user acceptance testing

static integration testing

white-box end-to-end testing

combinations

dynamic functional whitebox regression mobile UI exploratory pair testing

combinations

combine testing types and see what happens

is the result

obvious?
curious?
weird?
wrong?
... ?

Wittgenstein

Wittgenstein in 1947

Wittgenstein

1929: returns to Cambridge, starts lecturing in 1930

1936-1937: writes most of part I of Philosophical Investigations

1938: becomes German citizen (Anschluss), applies for British citizenship

1939: elected Professor of Philosophy at Cambridge (resigns in 1947)

1941-1944: works as a hospital porter and as a laboratory assistant

1947-1949: writes Part II of Philosophical Investigations

28 April 1951: dies in Cambridge (aged 62)

1953: Philosophical Investigations published

the problem with the Tractatus

the problem

The statement that a point in the visual field has two different colours at the same time is a contradiction.
The logical product of two atomic propositions can neither be a tautology nor a contradiction.

the 'solution'

Not all atomic propositions are logically indepent of each other.

the result

The logical is no longer a clear boundary and the whole theory comes crashing down.

Philosophical Investigations

"[...] we are committed to a faulty view of language, one that thinks that, to every meaningful word there must correspond some object " - Ray Monk

five red apples (PI 1)
names of numbers, names of colours, names of objects

the beetle in the box (PI 293)
pain and private languages

a behavior theory of language

"[...] a radical break with the idea that language always functions in one way, always serves the same purpose: to convey thoughts - which may be about houses, pains, good and evil, or anything else you please." (PI 304)

"For a large class of cases of the employment of the word 'meaning' - though not for all - this word can be explained in this way: the meaning of a word is its use in language." (PI 43)

more Wittgenstein

How to read Wittgenstein
- Ray Monk

Wittgenstein, The duty of genius
- Ray Monk

Wittgenstein's Vienna
- Allan Janik and Stephen Toulmin

How to read Wittgenstein - Ray Monk

software development model

xkcd software development

software development model

V-model acceptance testing

V-model acceptance testing

acceptance test-driven development (ATDD)

ATDD

software development model

agile testing

developer testing

automated testing

software development model

testing types do not exist on their own

part of an approach to software development

social aspects

people talking

social aspects

pair testing

mob testing

crowd testing

social aspects

end-to-end testing

black-box testing

technical testing, automated testing

security testing

social aspects

language does not exist outside of the social

testing types affect how people interact

wrap-up

"X" testing

anything can be "X"

so the meaning of "X" testing can be anything

most of the time, meaning is usage

today's heuristics

  1. definitions
  2. opposites
  3. all the elements
  4. combinations
  5. software development method
  6. social aspects

conclusion

clarity of meaning matters in communication

if meaning is usage... clarifying trumps correcting

explore
engage in dialogue
ask "can you show me?"

thank you!

Sli.do – C089

Joep Schuurkes - @j19sch
j19sch.github.io