Just because research is ‘scholarly’ doesn’t mean it’s quality

Even professionals who use health information daily haven’t been trained to appraise research critically

Medical research: just because it is original research doesn’t mean it is of a uniform quality. Photograph: iStockphoto

Medical research: just because it is original research doesn’t mean it is of a uniform quality. Photograph: iStockphoto

 

Nowadays when you do a Google search for a medical topic it’s common to see a selection of scientific papers packaged under the banner “scholarly articles” among the suggested links. It’s great to see this; it acts as a counterbalance to any unreliable sites a search may throw up.

But just because it is original research doesn’t mean it is of a uniform quality. Even some professionals who use health information on a daily basis haven’t been trained to appraise research critically, so to expect the lay reader to be able to objectively weigh up the evidence is a big ask.

The biggest challenge for many of us is probably the statistical information in a scholarly paper. With Benjamin Disraeli’s remark about “lies, damn lies and statistics” ringing in our ears, any remaining interest in deciphering the research numbers will likely be diluted. Actually this kind of scepticism is a really good starting point when compiling a list of questions with which to tackle a scientific paper.

In the run-up to the recent Brexit vote, David Spiegelhalter, the Winton professor of the public understanding of risk at the University of Cambridge, spoke of his exasperation at the cavalier misuse of statistics in the debate.

Writing in the Guardian, he suggested a checklist with which to assess statistical “spin”. It’s something we could usefully adapt in our assessment of clinical research and the overhyped press releases that sometimes accompany publication.

The checklist includes watching out for the old chestnut of implying causation when research just shows an association between two measurements. A recent example was research showing an association between weekend death rates in UK hospitals and reduced staffing; it quickly became a false “truism” that low numbers of staff in hospitals caused higher weekend death rates.

Proportions

The good news is experts at the Social and Public Health Sciences Unit at the University of Glasgow have launched an online tool called Understanding Health Research (understandinghealthresearch.organ you hyperlink, please) to guide anybody who wants to understand a health research paper through the process of asking the right questions.

Firmly aimed at non-scientists, it guides users through reading, interpreting and evaluating health research papers. As the quality of different types of research must be assessed in different ways, the tool helps users to identify and ask the right questions appropriate to each of the major types of health study.

For example, if you are trying to understand a clinical trial of a new drug, the Understanding Health Research tool will ask “Did the trial include a control group?” alongside an explanation of control groups. It then explains what the ramifications of your answer might be for the reliability and usefulness of the research. At the end of the tool, a summary page lists all the relevant positive and negative feedback to help you come to your own conclusions.

The originators say they were motivated to create the online resource by a lack of critical appraisal tools aimed at a general audience. “We decided to develop something that could help guide people through the process of understanding a research paper, demystifying important scientific concepts such as evidence hierarchies, scientific uncertainty and the difference between correlation and causation,” they write.

While tested with many different types of user to ensure that it strikes the right balance of accessibility and usefulness, the tool’s originators are keen to get more feedback.

Health literacy is the ability to understand, assess and use health information. Developing these skills will mean better relationships with healthcare providers, and better decision making about your health.

mhouston@irishtimes.com

The Irish Times Logo
Commenting on The Irish Times has changed. To comment you must now be an Irish Times subscriber.
SUBSCRIBE
GO BACK
Error Image
The account details entered are not currently associated with an Irish Times subscription. Please subscribe to sign in to comment.
Comment Sign In

Forgot password?
The Irish Times Logo
Thank you
You should receive instructions for resetting your password. When you have reset your password, you can Sign In.
The Irish Times Logo
Please choose a screen name. This name will appear beside any comments you post. Your screen name should follow the standards set out in our community standards.
Screen Name Selection

Hello

Please choose a screen name. This name will appear beside any comments you post. Your screen name should follow the standards set out in our community standards.

The Irish Times Logo
Commenting on The Irish Times has changed. To comment you must now be an Irish Times subscriber.
SUBSCRIBE
Forgot Password
Please enter your email address so we can send you a link to reset your password.

Sign In

Your Comments
We reserve the right to remove any content at any time from this Community, including without limitation if it violates the Community Standards. We ask that you report content that you in good faith believe violates the above rules by clicking the Flag link next to the offending comment or by filling out this form. New comments are only accepted for 3 days from the date of publication.