Use Consistency Oracles For Test Reporting

In the previous post, we presented consistency heuristics and gave an example of how they could be used as testing ideas on the Mac Disk Utility tool. This is not the only usage of consistency heuristics. They represent great help when the tester writes bug reports. The post is aligned with the Black Box Software Testing Foundations course (BBST) designed by Rebecca Fiedler, Cem Kaner, and James Bach.

You are using heuristic consistency with a comparable products. In the previous post where we applied consistency heuristics on Mac disk utility. On screenshot 4 we have available disk space shown in Disk Utility, and in the screenshot below, we have df output, as the comparable product. We discovered that df is using a different standard for disk size metric. This seems inappropriate. Let’s say that you are a Junior software tester and that your stakeholder is tester manager. How can you explain to the test manager that this could be bad?

After your investigation, you found a compelling reason why we have two metrics for file sizes. G is SI metrics and stands for 10×9, and Gi is IEC and stands for 2×30. Yes, those are math number systems. Let’s say that you do not have a clue what ara math number systems. But in your investigation, you found out that the hard drive of 80GB, after format, is shown in the operating system as 74.4 Gb. There was a lawsuit against Western digital! The reason was that the operating system is using the IEC standard for disk size metrics, and Western Digital is using the SI metric system. Because that system shows greater values.

df is a UNIX utility and it uses the IEC standard metric. Disk Utility is an Apple product, and Apple also sells hardware. My MacBook Pro is advertised as 256 GB. That number less in operating system GiB metrics. Do we have a problem here? Possible law suite? I would write a bug report because the GB metric gives the impression to customers that they will get more disk space for their money.

Let’s crack the numbers. Disk Utility shows 93.55 GB, df shows 80GiB.

1GiB ≈ 1.074GB[ wikipedia]

80 x 1.074 = 85.92. This still does not adds correctly. Wait, Disk Utility says 93.55–7.75 purgeable space = 85.8.

Tie the reports to the facts and data you collected in your research.

Originally published at https://blog.tentamen.eu on August 19, 2019.

Written by

Founder of Tentamen, software testing agency.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store