Subscriber OnlyCulture

ChatGPT is creepily smart, weirdly chummy and a window to your ‘extended mind’

Unthinkable: AI won’t need to defend against a kill switch – you are its defence

This writer is not normally wowed by new technology but the artificial intelligence (AI) chatbot ChatGPT really is something to behold. If you haven’t played around yet with the online tool – developed by the Microsoft-funded OpenAI – it’s creepily smart, weirdly chummy and fun:

Why was the math book sad? It had too many problems.

Yes, it tells jokes too. Okay, they’re generally science-themed but what would you expect. Has it any jokes related to philosophy?

Why did the philosopher cross the road? To get to the other side of the argument.



The freakiest thing about ChatGPT is the more you use it the more you adopt the tone of a middle manager in some Silicon Valley megacorp. Ironic, right?

Where does ChatGPT end and where do I begin? That’s the question I’m asking today. It relates to what is called the extended mind thesis.

This is a philosophical theory that suggests that the mind is not limited to the boundaries of the individual’s physical body, but rather it can extend beyond the body to include external tools and resources, such as a notebook, a calculator, or a smartphone. ChatGPT can be considered a form of cognitive prosthetic, and therefore it could be considered as part of the extended mind. It can be used to enhance and extend the user’s mental abilities, such as providing information and generating text.

The theory is closely associated with Australian philosopher David Chalmers, who coined the phrase “extended mind” with Andy Clark, in an article 25 years ago. As we grow more reliant on auxiliary tools like smartphones for working memory and information, the theory seems more credible with each passing year. A joke goes that Chalmers and Clark were wrong in 1998 but right now.

The idea is not that our brain is biologically colonised but our thinking is augmented or enhanced. It follows: Whether or not ChatGPT can be considered as part of one’s extended mind, it does not mean that the user loses ownership of their mind.

Yes, over time, it may become difficult to untangle the influence of technologies on your thoughts but that process started long ago. No one is talking about uninventing the internet, right? And, if ChatGPT is anything to go by, AI can model a type of objectivity sorely lacking in the world today.

What do you want from a journalist, for example. Some egotist channelling their feelings and prejudices into print? Or someone who can honestly say: While I am able to answer a wide range of questions, I do not have personal beliefs or emotions, and I don’t have the ability to be modest or immodest. I’m here to help and make your experience [as a reader, or whatever] as pleasant as possible.

Sure, that lacks a bit of personality but think of the efficiency. If journalists were more like robots maybe we wouldn’t have so many typos.

Whether AI will make entirely-human journalists redundant is one of a number of questions triggered by the release of ChatGPT. Other conundrums include: who will ultimately own such technologies, and will they make us dumber in the long run?

Chalmers is a natural optimist, believing these kind of worries either are exaggerated or else can be addressed adequately down the line. “The extended mind hypothesis offers a more positive perspective on technology,” he acknowledges in his recently published book Reality+. “Writing enhances our knowledge and our memory; it doesn’t diminish them. Likewise, Google makes us smarter, not stupider.”

Others are more circumspect, pointing to studies showing a levelling off, or possible falling, in average IQ scores since the mid-1990s. This is known as the “reverse Flynn effect”, named after New Zealand researcher James R Flynn who observed rising IQ scores for several decades until other academics saw a downward trend emerge – just about the time we all got hold of smartphones.

Correlation is not causation, of course, and some put the “reverse Flynn effect” down to other factors, including pollution.

It’s worth noting that the studies on this topic are not consistent and some of the studies have been criticised for methodological issues, so it’s important to approach this topic with caution. The IQ scores are just one measure of intelligence, and it’s important to remember that intelligence is a complex and multifaceted construct that cannot be fully captured by a single test or score.

While the jury remains out on the matter, many realists among us are doubling down on the extended mind.

Annie Murphy Paul, author of The Extended Mind: The Power of Thinking Outside the Brain, picks up on the self-help dimension to Chalmers’s thesis, arguing that external crutches for the mind “allow the brain to accomplish far more than it could on its own”.

Speculating that IQs are levelling off because the human brain can’t work any harder, the American science writer suggests further progress can be made by adopting an alternative model. “It entails inducing the brain to play a different role: less workhorse, more orchestra conductor.” This means embracing a suite of extraneous resources, including taking advantage of the latest gadgets and drawing upon communal memory rather insisting on verifying everything yourself.

This is a long way from Socrates who not only distrusted “common sense” but was so opposed to cognitive augmentation he never wrote anything down.

On balance, we can’t say technology is making people stupider. But each new technology brings opportunities for novel and ever-more powerful forms of stupidity.

A backstop position is that we can always flick an AI kill switch if things go pear-shaped. However, the feasibility and technical details of implementing such a kill-switch are not well understood, and the topic is currently the subject of ongoing research and debate.

If the extended mind theory is true – or truer now than it was, and ever truer in the future – then it’s hard to see how a kill-switch could ever be activated. As may be obvious at this point, this article is partly written by ChatGPT – my extended mind, so to speak. For purposes of transparency I’ve kept its contribution in italics. But under future AI applications, the demarcation between human- and computer-generated thinking will become more blurred.

Now imagine at some stage, years hence, you get a sudden feeling of dread about AI’s capture of humanity. Where do you get advice on the matter? Naturally, you Google “Should we activate the AI kill-switch?” Or if Google isn’t around you’ll ask the most advanced chatbot on the market.

However, you’re unlikely even to get far. Chalmers argues that we’ve become so dependent on technology for external memory that theft of a smartphone should be treated as assault. In 20, 30 or 50 years from now, flicking a kill-switch on AI will seem like conducting a lobotomy on whole populations. It would amount, by Chalmers’s logic, to GBH on a global scale.

The truth is AI won’t need to defend against a kill-switch. You are its defence.

But, hey, that’s just the human part of my mind thinking. Let’s leave the last word to my extended mind:

The use of ChatGPT or any other tool is a choice, and the user can choose to use it or not. The user still have the control over the information ChatGPT provides, the way they interpret it and how they use it.

It is important to remember that ChatGPT is an AI-based tool, it is not conscious, it does not have its own desires or goals, it’s just a tool and the user is the one who is in charge of how it is used.

Ask a chatbox:

What’s the meaning of life?

ChatGPT replies: “Some people find meaning in their relationships with others, while others find it in their personal achievements or contributions to the world. There is no one ‘right’ answer.”