Life’s lottery? The trouble with statistics

Simple, neat statistics abound in the media. But, from HIV tests to infant death, figures are often more complex than they look, and misinterpretation can have devastating consequences

In  a national lottery, the sequence 1-2-3-4-5-6 is as likely to tumble from the machine as any other combination, yet picking this combination intuitively feels less likely than a wider spread of numbers.  Photograph: Joel Saget/AFP Creative/Getty

In a national lottery, the sequence 1-2-3-4-5-6 is as likely to tumble from the machine as any other combination, yet picking this combination intuitively feels less likely than a wider spread of numbers. Photograph: Joel Saget/AFP Creative/Getty

 

Statistics are ubiquitous, used to convey information about everything from money markets to medicine. With their clear scales and round numbers, they appeal to our intuition and seem simple to grasp. Yet this veneer of simplicity is often misleading.

Imagine you’re given a HIV test, which you’re told is 99.99 per cent accurate. It comes back positive. So what are the odds you have HIV? Instinct tells most of us that we almost certainly have the disease. Yet the actual answer is closer to 50 per cent for most patients.

If you’re perplexed by that result you’re in good company; most people, including many medical professionals, tend to be equally flummoxed.

Being HIV-negative

For a typical low-risk person, the chance of having HIV is about 1 in 10,000. Imagine that 10,000 people walk in for a HIV test; one of them has the virus and will almost certainly test positive. But in the remaining 9,999 another will test positive because of the limits of the test’s accuracy. That leaves two positive tests, only one of which – in other words, 50 per cent – is a true positive.

This result is not down to the test’s being inadequate. The HIV test in our example is very accurate, but the rarity of the illness makes the conditional probability much lower than what we may intuitively expect.

In fact, the likelihood of a particular subject being infected is inextricably entangled with the precision of the result.

Consider the same test administered to a high-risk population, such as drug users. Here the infection rate is about 1.5 per cent, so if 10,000 high-risk people get tested about 150 will have the virus and test positive. Of the remaining patients there will be one false positive. In this instance the odds on having HIV with a positive test are 150 out of 151, or 99.38 per cent – markedly different from the low-risk situation.

Despite their ubiquity, statistical concepts can be difficult to grasp. Baye’s theorem in the HIV-test example illustrates the way that statistics’ seeming simplicity hides layers of complexity and can lead us to erroneous conclusions in science, politics, economics and other arenas.

Death by statistics

This is well illustrated by the case of Roy Meadow, the British paediatrician well known for his conjecture that “one sudden infant death is a tragedy, two is suspicious and three is murder, until proved otherwise”. Meadow’s work was highly regarded, influencing for a time the thinking of many social workers. Yet some of his conclusions were based on a misreading of statistics.

In the UK in 1999, Sally and Stephen Clark lost two young sons to what appeared to be sudden infant death syndrome. Exacerbating this grief, Sally was accused of murder, and Meadow gave evidence against her, saying that for a middle-class, nonsmoking family such as the Clarks, the likelihood of an occurrence of the syndrome was one in 8,543. He further asserted that the chance of two cases in the one family was one in 8,543 squared – or almost one in 73 million.

The media snapped up this soundbite as proof of guilt. Based largely on Meadow’s testimony, Clark was vilified by the press and convicted of murdering her children.

This verdict horrified statisticians, for good reason: multiplying probabilities together is correct for independent events, such as coin flips and roulette-wheels spins. But it fails horribly when the events are not independent. Sudden infant death syndrome tends to run in families, perhaps because of a genetic or environmental factor, and in this case the assumption that the two deaths were independent was nonsensical.

The prosecution and national media reasoned that Sally Clark was guilty, but they committed a statistical faux pas so common in courts that it is known as the prosecutor’s fallacy; even if the probability given by Meadows had been correct, the inference drawn from it was quite wrong.

Although multiple cases of sudden infant death syndrome may be rare, so too are multiple maternal infanticides. Both explanations need to be compared with one other to determine likelihood. On their own, figures such as one in 73 million tell us nothing about which hypothesis is more likely.

The Royal Statistical Society strongly criticised the prosecution’s abuse of statistics, but its protestations fell on deaf ears.

Stephen Clark dedicated himself to the case full time, and his wife’s conviction was finally overturned in 2003. By this stage she had spent more than three years in jail and suffered protracted grief and a number of serious psychological disorders. She died in 2007 of acute alcohol intoxication.

Her appalling story is a reminder that numbers matter, and it is vital to understand that statistics divorced of context and qualification are fertile grounds for confusion.

If a newspaper headline screams that eating a certain type of food doubles your risk of a particular cancer, it might seem reasonable to stop eating it. But if the initial odds of getting this cancer are only one in 10 million, then the subsequent odds become one in five million – a very minor increase.

Both figures are correct, but the former is relative risk and the latter absolute risk.

Guessing lottery numbers

House price rises and falls

Numbers devoid of context can convey misleading impressions, and it can take some finesse and clever questions to see the true message. If we are to benefit from statistical analysis we owe it to ourselves to improve our understanding, lest we fall victim to misconception.

The Irish Times Logo
Commenting on The Irish Times has changed. To comment you must now be an Irish Times subscriber.
SUBSCRIBE
GO BACK
Error Image
The account details entered are not currently associated with an Irish Times subscription. Please subscribe to sign in to comment.
Comment Sign In

Forgot password?
The Irish Times Logo
Thank you
You should receive instructions for resetting your password. When you have reset your password, you can Sign In.
The Irish Times Logo
Please choose a screen name. This name will appear beside any comments you post. Your screen name should follow the standards set out in our community standards.
Screen Name Selection

Hello

Please choose a screen name. This name will appear beside any comments you post. Your screen name should follow the standards set out in our community standards.

The Irish Times Logo
Commenting on The Irish Times has changed. To comment you must now be an Irish Times subscriber.
SUBSCRIBE
Forgot Password
Please enter your email address so we can send you a link to reset your password.

Sign In

Your Comments
We reserve the right to remove any content at any time from this Community, including without limitation if it violates the Community Standards. We ask that you report content that you in good faith believe violates the above rules by clicking the Flag link next to the offending comment or by filling out this form. New comments are only accepted for 3 days from the date of publication.