William Reville: Something has gone very wrong with science

The modern world depends on science, but as much as half of the literature may not be fit for purpose

Chief concerns with the way scientists do research include inadequate understanding of statistics, publishing studies that can’t be reproduced because they used shoddy procedures or didn’t have enough data, or because they ‘spun’ the data to please sponsors. Photograph: Thinkstock

Chief concerns with the way scientists do research include inadequate understanding of statistics, publishing studies that can’t be reproduced because they used shoddy procedures or didn’t have enough data, or because they ‘spun’ the data to please sponsors. Photograph: Thinkstock

 

The1960 Nobel laureate in physiology or medicine, Peter Medawar, famously remarked in 1983: “In terms of fulfilment of declared intentions, science is incomparably the most successful enterprise that human beings have ever engaged upon.”

Compare Medawar’s sentiment with the 2009 statement made by Marcia Angell, former editor of the New England Journal of Medicine and quoted in PLOS Medicine in October 2010: “It is simply no longer possible to believe much of the clinical research that is published or to rely on the judgment of trusted physicians or authoritative medical guidelines.”

In 1983, looking back on the history of science since the 17th century, we could all justifiably salute Medawar’s sentiment, but today we must face up to the fact that in recent times something has gone fundamentally wrong with science, perhaps our greatest human creation. On April 1st and 2nd, a UK meeting was organised by the Academy of Medical Sciences, the Medical Research Council, the Biotechnology and Biological Sciences Research Council and the Wellcome Trust to discuss this problem, specifically as it applies in biomedical research.

The Chatham House rule was applied at this meeting; that is, participants are free to use the information they receive but neither the identity nor affiliation of any speaker nor any other participant must be revealed.

Richard Horton, editor-in-chief of the Lancet, attended the meeting and wrote a memorable account in the journal (April 11th), illustrated by the following quote: “The case against science is straightforward: much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn towards darkness. As one participant put it, ‘poor methods get results’ . . .

“The apparent endemicity of bad research behaviour is alarming. In their quest for telling a compelling story, scientists too often sculpt data to fit their preferred theory of the world. Or they retrofit hypotheses to fit their data. Journal editors deserve their fair share of criticism too. We aid and abet the worst behaviours. Our acquiescence to the impact factor fuels an unhealthy competition to win a place in a select few journals. Our love of ‘significance’ pollutes the literature with many a statistical fairy-tale. We reject important confirmations. Journals are not the only miscreants. Universities are in a perpetual struggle for money and talent, endpoints that foster reductive metrics, such as high-impact publication . . . and individual scientists, including their most senior leaders, do little to alter a research culture that occasionally veers close to misconduct.”

 

Shoddy procedures 

John Ioannidis, director of the Meta-Research Innovation Centre at Stanford, is a respected critic of science. His chief concerns with the way scientists do research include their inadequate understanding of statistics, publishing studies that can’t be reproduced because they used shoddy procedures or didn’t have enough data, or because they “spun” the data to please sponsors. University scientists are compelled to publish prolifically, which favours quantity over quality, and the pre-publication peer-review system is no longer fit for purpose.

Much of this could be tackled effectively by improving the training of scientists. Young scientists receive little formal training in how to carry out reliable research. You spend undergraduate years studying the established body of knowledge in your subject and you learn how to research only later, during your PhD project, when you are apprenticed to an established scientist. But this latter learning is an informal, hit-and-miss affair, largely absorbed by osmosis. All science students should have to undertake rigorous coursework on how to carry out effective scientific research.

Neither is there any formal code of ethics in science. Ethics should be taught at undergraduate and graduate levels and a code of ethics should be assented to by graduates at all graduation ceremonies.

We can no longer doubt that a crisis looms in science. The modern world is utterly dependent on science, so this crisis must be resolved. That great cathedral of scientific progress, the peer-reviewed scientific literature, is beginning to crumble. We either restore the cathedral or we will soon stand in its rubble.

The Irish Times Logo
Commenting on The Irish Times has changed. To comment you must now be an Irish Times subscriber.
SUBSCRIBE
GO BACK
Error Image
The account details entered are not currently associated with an Irish Times subscription. Please subscribe to sign in to comment.
Comment Sign In

Forgot password?
The Irish Times Logo
Thank you
You should receive instructions for resetting your password. When you have reset your password, you can Sign In.
The Irish Times Logo
Please choose a screen name. This name will appear beside any comments you post. Your screen name should follow the standards set out in our community standards.
Screen Name Selection

Hello

Please choose a screen name. This name will appear beside any comments you post. Your screen name should follow the standards set out in our community standards.

The Irish Times Logo
Commenting on The Irish Times has changed. To comment you must now be an Irish Times subscriber.
SUBSCRIBE
Forgot Password
Please enter your email address so we can send you a link to reset your password.

Sign In

Your Comments
We reserve the right to remove any content at any time from this Community, including without limitation if it violates the Community Standards. We ask that you report content that you in good faith believe violates the above rules by clicking the Flag link next to the offending comment or by filling out this form. New comments are only accepted for 3 days from the date of publication.