Fifteen Brainstorming Test Idea Challenges — Tentamen Software Testing Blog

Image by Elias Sch. from Pixabay

TL;DR

Now that we know a good testing taxonomy’s properties using this excellent paper: Bug Taxonomies: Use Them to Generate Better Tests [Vijayaraghavan, Kaner]. Let’s brainstorm fifteen brainstorming test ideas challenges. Many thanks to Marcel, who sublimed this great resource on his blog, That’s the buffet table.

Challenges

In the BBST Test Design Course by AST, there is an exercise to list risks related to variable values. Students must pick a variable suitable for equivalent class analysis. Context is Open Office application. Students often confuse risks with test ideas. And this is proof that creating a test idea is a challenging task.

As there are many test techniques, it is very easy to lost focus. For example, we need to brainstorm test ideas using Specification Analysis. We have explicit and implicit specifications. And it is very easy to lost focus by brainstorming implicit specifications because we need to guess which implicit specifications are important for a client.

Using the same example, specification testing. To produce high-quality specification takes great skill. If I could get a penny for all the logical errors that I have found in specifications.

Software testers have limited time. Because of so many test design ideas, it is hard to a trade-off between time and coverage of test ideas.

If you have never heard of SFDEPOT or HTSM, your test ideas probably missed many important issues. Those two are examples of many software testing frameworks.

This is the case when several “different” test ideas are the same:

Two tests are distinct if one test would expose a bug that other tests would miss. It is not so obvious, but if you run all distinct tests, you are sure that there are no bugs left in the software [Kaner].

For example, I am testing complex software that is a web application, several users could log in, but only one user could run complex calculations simultaneously. Management wants load testing to see how many users could use the application. I know that result is one user without any need for load testing.

To good test ideas, I must have one main testing information objective and second testing information objective. Having more information objective would produce low-quality test ideas.

Basically, you are stuck. Maybe you are done with testing, and the product is ready?

Even with software testing frameworks, we still need to be creative to create test ideas with high-quality properties.

What obstacles will we meet during the testing?

The issue or the problem that we have with our product. This is the information objective of our testing activity.

Lateral Thinking is, simply put, the skill to think outside the box.

We based our test ideas on A Model of a product under test. This model is partial or wrong to some extent. But without a model, we are in trouble. For example, we could not know when we depleted test ideas for a model.

Remember when the test idea is unique; if it finds a bug, that bug could only be detected using this testing idea.

This is the trap when we create a lot of test ideas without “catching” product broadness. For example, we have many functional-based test ideas but very few scenarios based on test ideas.

Originally published at https://blog.tentamen.eu on November 24, 2020.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Karlo Smid

Karlo Smid

Founder of Tentamen, software testing agency.