'Good enough' is the benchmark for software testers, but what constitutesgood enough? Karlin Lillington talks to two Irish-based experts on when software is ready to ship.

Software testers are the clampers of the software world: everyone appreciates that what they do is usually helpful and necessary, but no one likes to be on the receiving end when they do their job well.

Testers are the people who take the code written by software developers and pick through it carefully, hunting down the errors, bugs, glitches - whatever you want to call the smorgasbord of problems that could keep a program from performing well when a user eventually loads it into a computer.

"One of our jobs as testers is to tell people their baby is ugly. So it's no wonder we're not very well liked," admits a somewhat tongue-in-cheek Mr Lee Copeland, a software developer and tester who works as a consultant and speaks widely on software testing.

The audience he is addressing, all software testers working for companies in the Republic, laughs in agreement. Members of SoftTest (www.softest.ie), an Irish software testers networking group, they know exactly what it's like to have to tell developers and management that a program can't ship because chunks of it need to be recoded.

And, more frustratingly, they know what it is like to have management tell them software is going to ship anyway, whether they feel it is ready to go or not. Judging by audience conversation, few things are more aggravating in the tester's world than having a product going out the door that wouldn't get their seal of approval.

SoftTest members do a cautious, precise and time-consuming job that aspires to but doesn't actually achieve perfection, according to Mr Copeland and fellow speaker, Dr David Parnas, professor of software engineering and director of the Software Quality Research Laboratory at University of Limerick. The two were featured at SoftTest's semi-annual seminar in Dublin this week.

Testers work in the world of "good enough" - the point at which a program has no major problems and can be shipped, but is almost never error-free, says Mr Copeland.

But determining the point at which "good enough" has been reached can be extremely difficult, not least because different groups will pick different definitions of good enough, he says.

In a talk on how to figure out when to stop testing, he suggested several different approaches:

one can go for a consensus among the parties involved in developing the software;

one can set predetermined criteria and stop testing when those are met;

one can base a decision on whether costs are beginning to climb beyond an acceptable point;

one can stop testing when the discovery rate of defects drops to an agreed level.

Of course, the most persuasive of all arguments for stopping, if also the most annoying, is when a manager halts the testing process and says: "Ship it", Mr Copeland says.

But the problems with each of these methods is the context. How does a tester decide on the predetermined criteria when it is hard to know if one has chosen the right criteria, or all the necessary criteria? Defects and bugs also tend to be hidden and tricky to find, Mr Copeland says, so determining when you've found the worst of them is no easy task.

Testers can sometimes look to their project manager in making the "good enough" decision, he says. "A good project manager can weigh up all these elements and determine when the balance tips, so that the advantages outweigh the disadvantages to shipping."

But the individual tester still needs to decide on similar trigger points for all the criteria that go into testing. "There's no single criterion to decide when to stop," Mr Copeland says.

Dr Parnas came at the same question from a different standpoint. He is interested in how the software development and testing can be better structured and, at the same time, documented.

"Why is software so hard to test?" he asks. "Because it's unstructured, it's extremely sensitive to minor errors and there's no such thing as 'almost right'. 'Almost right' means it's wrong," he argues.

He believes "reliability is the only quality where you can statistically decide when to stop". But measuring reliability means lots of testing, and, ideally, automated testing.

His answer is to start with good documentation before any coding begins.

His own method of teaching students is to require them to learn to write careful specifications for each component of a software program. The specifications are written out in easy-to-read "function tables".

The code is then written to accord with the specifications laid out in the tables. Because the tables describe functions, a developer can write code in any number of computer languages to perform the functions.

A tester can begin checking the function tables even before any code is written, hastening the testing process, Dr Parnas says. And once the program is available, it is much simpler to test whether the software does what is laid out in the specification tables, than it is trying to guess what the programmer was trying to achieve, or relying on documentation written after the program is created.

"Developers should invest a little time in doing something like this because it actually shortens the development time. It makes you think, it's more easily read and checked by other people than code, and it gives you testing tools," Dr Parnas says.

And the specifications also serve as the documentation for the software - documentation that "has coffee rings on it because it's used. Nobody trusts today's documentation because it's vague, incomplete, inconsistent and inaccurate", he says.

In other words, writing programs to fit function tables is a way of easing the tester's job, enabling the tester to do a better, more accurate, yet faster job, he says.

The problem for testers, alas, is to get the developers within their companies to use the table method. While many in the audience were intrigued by Dr Parnas's arguments, they showed some doubt about getting the developers to do things in a way that would benefit the testers.

The push for table-based development would have to come from management - and they're the ones telling them to just ship it. A tester's life is not an easy one.