Can AI be trusted? Yes, but bear in mind that it doesn’t think like us

The economic opportunities presented by generative artificial intelligence are now widely acknowledged. Among the more optimistic views is a November 2023 study of 2000 firms by Microsoft and IDC which suggests that for every dollar a firm spends in AI, the average return is three and half times that figure. Organisations are realising a return on their AI investments within 14 months, the research claims, and IDC projects that generative AI will add nearly $10 trillion to global GDP over the next 10 years.

Nonetheless, concerns remain about the limitations of AI, specifically about how much the existing technology can be trusted. The habit of the ChatGPT application to “hallucinate” or, to put it more accurately, take a wild guess when it doesn’t know the answer, has been well documented.

If generative AI can’t be trusted at the controls and needs a human sitting alongside it to ensure the efficacy and accuracy of its work, is there a danger that much of the promised productivity gains will fail to be realised?

Geraldine Magnier of Idiro Analytics, which specialises in AI-based solutions, agrees that hallucination has been a problem in early iterations of generative AI and that the technology has its limitations. However, she thinks users need to build this into their expectations.

Now is the time for a cohesive, age friendly healthcare system

---

How your legacy gift to Concern Worldwide leaves a better future for some of the world’s poorest families

---

Good governance is critical for effective healthcare investigations

---

Joined-up IT means medics have more time to focus on patient care

---

“It’s striving to do its best by you and it does not have a rational mind,” says Magnier. “It can be likened to an over-enthusiastic intern. It does take away a lot of the donkey work if you are writing a report, for example, but it does need to be fact checked. There are now a myriad of IT tools that will allow you to that fact checking.”

A healthier way of looking at generative AI, she says, is to view it as a second brain, working alongside you on a task. “Both brains will know different things. The AI might be very strong on the factual side but it might be operating off old data and the context may not be right. Human oversight and critical thinking remain vital.”

Owen Lewis of KPMG agrees. “Use the capability as an efficiency play but don’t delegate quality and human judgement to an algorithm that by its nature is making a series of approximations, albeit very intelligent-sounding approximations,” he says.

Lewis’s point alludes to the seductive nature of generative AI. The fluidity and sophistication of the language it uses often serves to generate false confidence on the part of its users in the accuracy of the AI’s research output. It sounds like it knows its stuff, even when it doesn’t.

Users need to more specific in the questions they ask the AI, Lewis advises. “You need to be very clear which data sources it should use so, for example, you should ask it to look for tax information on revenue.ie and to label the references of where information is taken from so details can be checked.”

Prompt engineering is a huge help in harnessing the benefits of AI and is emerging as a huge sector in this field, notes Claire Carroll, an independent consultant who specialises in technology ethics. Some of the more recent applications in generative AI have also sought to provide more nuanced understanding of language. Claude, for example, developed by former Open AI head of research Dario Amodei’s firm, Anthropic, as an alternative to Chat GPT, claims to have superior qualities in the areas of context, intent and emotional tone, she notes.

Carroll acknowledges that it will take time to harness the benefits of AI and to figure out to what extent it can be fully trusted.

“There is a somewhat utopian picture around the efficiency that AI will provide,” she says. “The reality is that we will need to work quite hard at an oversight and governance level. The level of infrastructure that needs to be built around oversight is significant. It won’t be a free lunch.”

Erik O’Donovan, Ibec’s head of digital policy, says the new EU AI Act will support trust as it includes a requirement for human oversight of AI systems. “Limiting the purpose and responses of generative AI through safety and content control can also increase focus, effectiveness and minimise irrelevant results,” he says. “Curation is getting better over time.”

Brenda Jordan of Sobi Analytics, meanwhile, says that while people need to be patient with the imperfections of the technology while it is in learning phase, she is excited about what it can already do.

For example, in analysing a set of business data to forecast future performance outcomes, it can not only do a good job in predictive analytics, but can also provide prescriptive analytics.

“It can address questions such as ‘How can I change what’s going to happen in the future?’, ‘What are all the variables I need to take account of?’ and ‘What specific actions should I take to achieve desired outcomes?’”