Hardening Expectations for Software Quality

A hardware manifestation of a software bug, for example the Pentium FDIV error of 1994, is so rare that people can still remember that specific incident more than 10 years ago. In contrast, imagine saying "that Windows bug in 1994" (or, for that matter, "that Unix bug in 1994") and having anyone know exactly what you meant.

But complex semiconductors "are basically all software until you fabricate them," observed Susan Marie Kunz, founder and CEO of Solidware Technologies in Boulder, Colo., during a conversation we had last week. It seems odd, she said, that semiconductor quality stays abreast of rising complexity while enterprise platform and application software struggles to keep up.

Closing that gap is the goal of Solidware's debut product, Splat, introduced today and available for trial download from the company's site at www.solidw.com. The company's vision for the product, Kunz said, is to elevate software quality with the same strategies of aggressive automation, task prioritization and cost-effective risk reduction that she and her colleagues bring from their prior experience in semiconductors.

Growth of complexity and scale, Kunz said, "is emerging on the software side with open source and other developments that leave you not quite sure of code quality; modules increasingly aren't designed initially to work together, so use cases aren't comprehensive," she observed.

The problem of software complexity is compounded, Kunz added (and I emphatically agree), by the industry's rapid uptake of multithreaded hardware platforms and software strategies. "When you introduce multithreading, a lot of code will break," she said. "You don't have to make that move, but all the new CPU architectures … if you don't use it, performance will definitely be impacted. The companies that make their code work well in a multithreaded environment will definitely have an advantage over those that don't -- but it's definitely a risk, from a defect point of view."

A concurrent growth of complexity, she said, comes with the move from 32- to 64-bit systems. One application, she said, went from less than a CPU-year to more than 500 CPU-years of testing required to achieve comparable coverage.

Too many software testing tools, Kunz said, fail to live up to their potential because it's up to coders to keep track of the opportunities to use them. A tool may automate the performance of the test itself, but identification of when and what to test is no simple thing.

"There's no reason why a person should have to say, 'Oh, I messed with memory, I have to run Purify.' If I know you did that, and I know you have Purify, that should kick off automatically," Kunz urged. "We ask customers, 'What would make your life easier?' They so often say, 'Just tell me when I touch my code, what could I have broken?'"

I absolutely agree with that goal, and I hope that readers of this column will share with me their assessments of how close Solidware's Splat comes to achieving it -- and what else they're using toward that end.

Tell me what you're afraid of breaking at peter_coffee@ziffdavis.com.

This article was originally published on eWEEK.com.

Copyright © 2005 Ziff Davis Media Inc. All Rights Reserved. Originally appearing in Dev Source.