Difference Between Software Fault, Error And Failure — Tentamen Software Testing Blog

Image for post
Image for post
Elixir Union Function Implementation With A Fault

TL;DR

In the previous post, we commented on our answers for exercise Specification For Simple Function. Let’s move on and discuss how software fault, error, and failure differ from each other. We will introduce you to software testing based on the remarkable book, Introduction To Software Testing by Paul Ammann and Jeff Offutt.

Limitation

The theoretical limitation of software testing is that testing can not show the absence of failures.

The problem of finding all failures in the program is undecidable [Ammann and Offutt].

Many testing professionals consider a successful test a test that finds a failure. The problem with that statement is that this is level two software testing thinking.

Validation Vs. Verification

These software testing definitions are from the book Introduction Software Testing. I found them useful, but it is ok to use different definitions.

Validation is a process of evaluation software at the end of development to ensure compliance with intended usage. It is usually done by non-developers with strong domain knowledge.

Verification is the process of determining whether the products of a given phase of the software development process fulfills requirements established in the previous phase. This is more technical activity and usually is done by developers and technical testers.

It is essential to state that if verification passes, validation might still fail.

Fault, Error, And Failure

Software fault is a static defect in the software. Software failure is an incorrect internal software state that is a manifestation of software fault. Software failure is product external unexpected behavior concerning expected behavior (listed in requirements or common sense behavior). The fault is always a developer (human) software design mistake.

Example

Let’s examine our Elixir union function from exercise on a simple function specification. Here is our union function with error handling:

Let’s introduce a fault that will cause wrong error state, and failure for specific test data:

The developer made a software design mistake by relying on Elixir List concatenation operator ++ as union implementation and forgetting to add Enum.uniq() the function that removes duplicate List elements. That design mistake is a failure that causes error function state because there is only one union function state, final List that contains elements of both Lists.

Input test data [1, 2, 3] and [4, 5, 6] does not trigger failure because the result is actual union of those two lists: [1, 2, 3, 4, 5, 6].

But input test data [1, 2, 3] and [3, 4, 5] triggers failure because the result is [1, 2, 3, 3, 4, 5] but requirement states that union is [1, 2, 3, 4, 5 ].

Remember

Faults and failures could only be identified by developers, while failures could be caught both by developers and testers.

Originally published at https://blog.tentamen.eu on August 26, 2020.

Written by

Founder of Tentamen, software testing agency.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store