This post gives a definition of what is nontestable program. Software tester can still test such a program. This definition is aligned with the Black Box Software Testing Foundations course (BBST) created by Rebecca Fiedler, Cem Kaner, and James Bach.
In the previous post, we gave a definition of an oracle. But those oracle definitions come with two assumptions:
- There is a mechanism that helps the tester to determine the correctness of a system under test
- Tester can determine System Under Test correctness using oracle mechanism in a reasonable amount of time and a reasonable amount of effort
Program is considered non testable if:
- There is no oracle
- There is a theoretical oracle, but practically it is not possible to use it to determine program correctness.
By this definition we can conclude:
Many, if not most programs are by our definition considered non testable.
An example is Google Search.
When we describe testing as a process of comparing empirical results to expected results, we must consider that even the basic process of comparison requires human judgment, based on an understanding of the problem domain [BBST Foundations]
Originally published at https://blog.tentamen.eu on July 12, 2019.