No Terminator-like AI yet but the risk is from systems that seem to be conscious, neuroscientist Anil Seth

Leading neuroscientist Anil Seth’s book ‘Being You: A New Science of Consciousness’ is a Sunday Times bestseller that emerged from 20 years of research into the relationship between consciousness and the brain. He talks to Sunday Times about the future of AI
Your TED Talk ‘Your brain hallucinates your conscious reality’, has got more than 14 million views. Did you expect it to go viral?
It’s been an enormous surprise! I think one reason for its popularity is that consciousness is both one of the deepest mysteries in all of science and philosophy, and also one of the most personal. We’ve all wondered about where we were before we were born or what happens after we die. And as science has advanced, the enigma that the brain somehow is the seat of everything that makes life worth living, just seems more and more compelling.

What made you study consciousness?
I remember a time when I was young — maybe seven or eight — looking in the mirror and wondering what was staring back at me, and dimly realising that at some point in the future what I called ‘I’ would cease to exist.

I suppose I’ve been thinking about the mystery of consciousness in one way or another ever since then. But when I went to university in the early 1990s, to study natural sciences at Cambridge, consciousness was indeed off the menu. I remember being advised many times that pursuing an interest in consciousness would be a terrible career decision. What I didn’t know at the time was that a new science of consciousness was beginning to emerge, in a way that was well grounded in the mind and brain sciences as well as in philosophy. And so, after a PhD in artificial intelligence at the University of Sussex, I moved to California, to a place called The Neurosciences Institute where consciousness research was not only back on the menu, it was the main course. And I’ve never looked back.

How do you define consciousness? And why does it matter?
Consciousness is any kind of experience whatsoever — the redness of red, the painfulness of pain, the thirstiness of thirst. It is what makes us more than mere biological objects roaming around in the subjective dark. And this, of course, is why it matters. Without consciousness there would be no world, no self, no emotion, no agency — there would be nothing at all. Consciousness is what makes life worth living — and sometimes, seemingly, worth leaving. There is also a more philosophical way in which it matters. Something called the ‘sentience principle’ says that anything that has the capacity to be conscious has a moral status. If we accept this, and by and large we should, then understanding the nature of consciousness is absolutely fundamental to figuring out how we should behave in the world: how we should treat non-human animals, newborn infants, and perhaps even future machines.

Growing up as the son of an Indian father and British mother in England, what was your perception of yourself?
I grew up in South Oxfordshire, and besides a brief trip to the family home in Allahabad when I was six, my Indian heritage didn’t seem to feature much. But by my late teens I knew much more about both my Indian and my English heritage, and looking back now, I can see how formative this was. Living a life enriched by two cultures can inoculate you from taking how things seem for how they are. There is always another way, another view. And when in India, I almost feel as if I am a different person. This malleability of the self continues to fascinate and inspire me.

You recently asked ChatGPT to write your bio. Did it do a good job?

It was both surprisingly good and revealingly bad. It was right about fairly general things — what I do for a living, what my areas of expertise are within neuroscience — but it made mistakes about where I was born, where I studied for my PhD, and so on. When I asked it to redo it with fewer errors, it made things worse. At least in this current incarnation, AI really doesn’t understand or know anything at all.

Should we be worried about AI becoming conscious? (Asking for a friend)
There is plenty to worry about already such as the threats posed by misinformation, disinformation, biased decision making, image deepfakes and so on. We can no longer trust what we see or hear. And when AI algorithms are placed in control of things, we face the tricky problem of ‘value alignment’ — of making sure that an AI system acts for human or planetary benefit. This is turning out to be surprisingly challenging.

These worries are often bundled together with the general fear that AI will become conscious and then set out to enslave or destroy humanity, as in films like ‘Terminator’. But this is lazy thinking. Intelligence and consciousness are different things, and there is no reason to assume that consciousness will just come along for the ride as AI gets smarter. Some tech companies, though, are deliberately trying to build conscious machines. Here there is a lot more uncertainty. Since we don’t yet know what is necessary for consciousness, we can’t rule out the possibility of conscious AI, whether it happens on purpose or by accident. Right now, we need to worry about AI that merely gives us the appearance of being conscious, even if, under the hood, there is no consciousness happening. With language models like GPT we are almost at this stage already. This will also be highly disruptive, because our minds are not equipped to interact with systems that we think have consciousness but do not.