Fifteen Brainstorming Test Idea Challenges — Tentamen Software Testing Blog
--
TL;DR
Now that we know a good testing taxonomy’s properties using this excellent paper: Bug Taxonomies: Use Them to Generate Better Tests [Vijayaraghavan, Kaner]. Let’s brainstorm fifteen brainstorming test ideas challenges. Many thanks to Marcel, who sublimed this great resource on his blog, That’s the buffet table.
Challenges
In the BBST Test Design Course by AST, there is an exercise to list risks related to variable values. Students must pick a variable suitable for equivalent class analysis. Context is Open Office application. Students often confuse risks with test ideas. And this is proof that creating a test idea is a challenging task.
Lack Of Focus
As there are many test techniques, it is very easy to lost focus. For example, we need to brainstorm test ideas using Specification Analysis. We have explicit and implicit specifications. And it is very easy to lost focus by brainstorming implicit specifications because we need to guess which implicit specifications are important for a client.
Lack Of Clarity
Using the same example, specification testing. To produce high-quality specification takes great skill. If I could get a penny for all the logical errors that I have found in specifications.
Losing Time
Software testers have limited time. Because of so many test design ideas, it is hard to a trade-off between time and coverage of test ideas.
Lack Of Structured Framework
If you have never heard of SFDEPOT or HTSM, your test ideas probably missed many important issues. Those two are examples of many software testing frameworks.
Redundant Ideas
This is the case when several “different” test ideas are the same:
Two tests are distinct if one test would expose a bug that other tests would miss. It is not so obvious, but if you run all distinct tests, you are sure that there are no bugs left in the software [Kaner].
Unable To Eliminate Test Ideas That Do Not Fit
For example, I am testing complex software that is a web application, several users could log in, but only one user could run complex calculations simultaneously. Management wants load testing to see how many users could use the application. I know that result is one user without any need for load testing.
Unable To Locate A Central Idea
To good test ideas, I must have one main testing information objective and second testing information objective. Having more information objective would produce low-quality test ideas.
Idea Train Stops
Basically, you are stuck. Maybe you are done with testing, and the product is ready?
Unable To Inspire Creativity
Even with software testing frameworks, we still need to be creative to create test ideas with high-quality properties.
Unable To Identify The Challenge
What obstacles will we meet during the testing?
Unable To Define The Issue
The issue or the problem that we have with our product. This is the information objective of our testing activity.
Unable to Induce Lateral Thinking
Lateral Thinking is, simply put, the skill to think outside the box.
Lack of Paradigms
We based our test ideas on A Model of a product under test. This model is partial or wrong to some extent. But without a model, we are in trouble. For example, we could not know when we depleted test ideas for a model.
Ideas: Large Quantity And Of Low Quality
Remember when the test idea is unique; if it finds a bug, that bug could only be detected using this testing idea.
Lots Of Depth But No Breadth In The Ideas
This is the trap when we create a lot of test ideas without “catching” product broadness. For example, we have many functional-based test ideas but very few scenarios based on test ideas.
Originally published at https://blog.tentamen.eu on November 24, 2020.
