The authors of a controversial academic paper that claimed Irish judges use Wikipedia as a source for their rulings have published a new paper disputing suggestions that their research is flawed.
The original paper was published in 2022 and was criticised in a subsequent study published in May 2023, co-authored by High Court judge Mr Justice Richard Humphreys, which said citations in judgments are driven mainly by legal submissions and maintained there was no “Wikipedia effect”.
The authors disagreed with the study’s criticisms and on Monday published a detailed paper, entitled Trial by Internet – a response to judicial critics, standing over their research methodology.
In the original paper, researchers at Massachusetts Institute of Technology (MIT) and Maynooth University arranged for 154 articles on Irish Supreme Court decisions to be written by law students. Half were uploaded to Wikipedia and the rest were kept offline as a control group.
Taco Bell’s arrival could add spice to Ireland’s love affair with American fast food
‘There is no justice for this’: Driver in car crash that led to deaths of Monaghan teenagers jailed for seven years
Who felt informed enough in those first few days to call Alan Hawe a cold-blooded murderer?
‘Say Kate is your boss’: Pharmacist claims ex-TD O’Connell and husband were working him ‘to death’
The authors said the cases mentioned in the uploaded articles were 20 per cent more likely to be cited as precedents in court rulings than similar cases that had not been uploaded.
They also claimed an analysis of the language used in the judgments found the “linguistic fingerprints” of the Wikipedia articles.
The later research, carried out by Mr Justice Humphreys and six judicial assistants who worked with judges of the superior courts, criticised the “headline-grabbing claims” in the 2022 paper.
Mr Justice Humphreys said there were “fundamental problems” with that paper rendering its conclusions unreliable and that “baseless speculations” seemed to come from a lack of knowledge about how judgments are produced.
It was claimed the 2022 paper worked off a commercial website rather than the official courts.ie website where judgments are published, leading to the possibility of consistency issues.
Among other criticisms, it argued the authors compared too long a period of citations before the articles appeared on Wikipedia with a short period of about two years afterwards.
Using different methodology to compare citations of the cases mentioned on Wikipedia and those in the control group, the 2023 paper said the citations rose by the same amount and that meant no effect from the cases being mentioned on Wikipedia.
In response to that research, one of the original authors of the 2022 paper, Dr Brian Flanagan, an associate professor at the school of law and criminology at Maynooth University, said their analysis stood up to the criticism levelled at it.
In their latest paper, the original authors say their use of commercial data was cross-checked against the court’s own records and a small difference of less than 1 per cent did not change their conclusions and was not a research “flaw”.
They said further analysis provided support for their findings that cases added to Wikipedia jumped immediately in terms of citations by judges after being added.
Despite claiming to “debunk” their research, the 2023 paper did not do the work necessary for that, they said. What the 2023 paper did instead was to attempt “an entirely different analysis, with far less data and supporting argument”.
They said a “notable benefit” of the type of analysis in their 2022 paper – a randomised control trial – “is that it can establish causation without having to rule out all other possibilities”.
This was why such analysis is used in medical trials “and why, when a drug is approved for treatment, no one objects by saying not all other medical explanations were checked”.
The 2023 paper “read more like legal submissions than social science” and did not provide the research needed to show whether the critics’ objections would undercut the arguments advanced in the 2022 paper.
Their reframing of their research results as concerning “legal professionals” rather than “judges” was made in response to feedback, they also said.
“Some had interpreted our original wording as indicating that the discovered effect must stem from judges/assistants alone. We had always understood that it was likely that lawyers were also playing a role. Our rewording simply reflected the need to clarify that we were not limiting the mechanism to judges and their staff.”
