The idea of AI producing consciousness has long existed in science fiction - think back to the supercomputer-turned-villain HAL 9000 in the 1968 film 2001: A Space Odyssey - and as AI advances at a rapid pace, it's a possibility that's becoming less and less outlandish, and is even being endorsed by leading AI figures.
Last year, Ilya Sutskever, chief scientist at OpenAI, the company behind chatbot ChatGPT, tweeted that some of the most advanced AI networks could be 'slightly conscious'.
Many researchers say that AI systems have not yet reached the point of consciousness. But the speed of AI's evolution has left them pondering: how do we know if AI produces consciousness?
To answer that question, a team of 19 neuroscientists, philosophers, and computer scientists has come up with a list of criteria that, if met, would indicate that there's a high likelihood that a system is conscious. They released the provisional guidelines, which have not yet been peer-reviewed, in the arXiv preprint repository earlier this week.
Co-author Robert Long, a philosopher at the Center for AI Security in San Francisco, California, said they undertook the work because 'there seems to be a real lack of detailed, empirically-based, thoughtful discussion of AI consciousness'.
The team said the inability to identify whether AI systems are conscious could have significant ethical implications. Megan Peters, a neuroscientist and co-author of the study, said that if something is labeled as 'conscious', 'that must dramatically change the way we treat it as humans'.
Long added that, to his knowledge, companies building advanced AI systems are not putting enough effort into assessing the consciousness of these models and planning accordingly. He said, "Despite listening to some of the heads of leading labs, they do say that AI consciousness or AI sentience is something worth thinking about, I just don't think it's enough."
Nature contacted the two main tech companies pushing AI - Microsoft and Google. A spokeswoman for Microsoft said the company's development of AI centers on assisting humans to be more productive in a responsible way, rather than replicating human intelligence.
The spokesperson said that since the launch of GPT-4, the latest version of ChatGPT's public release, it's clear that "we're exploring how to realize the full potential of AI for the benefit of society as a whole, and that requires new ways to assess the capabilities of these AI models." Google, for its part, did not respond.
01. what is consciousness?
One of the challenges faced when studying AI consciousness is defining what consciousness is. Peters said that for the purposes of the report, the researchers focused on 'phenomenal consciousness', which is subjective experience. That is, the sensations that exist in a human, animal, or AI system system (if one of them is shown to be conscious).
There are many neuroscience-based theories describing the biological basis of consciousness. But there is no consensus on which one is the 'right' one. The authors therefore used a range of theories to create their framework. Their idea is that if an AI system functions in a way that matches multiple aspects of these theories, then it is more likely to be conscious.
They argue that this method of assessing consciousness is better than simply conducting behavioral tests, such as asking ChatGPT if it is conscious or challenging it and observing its responses. That's because AI systems have made incredible strides in mimicking humans.
Neuroscientist Anil Seth, director of the Center for Consciousness Science at the University of Sussex in the United Kingdom, believes that the team's theoretically rigorous approach is a good choice. However, he says, "We also need more precise, well-tested theories of consciousness."
02. a theory-intensive approach
To develop their criteria, the authors assume that consciousness has to do with how systems process information, regardless of whether they are made of neurons, computer chips, or other materials. This approach is known as computational functionalism. They also hypothesized that neuroscience-based theories of consciousness, which are derived from brain scans and other technical studies of humans and animals, could be applied to AI.
Based on these assumptions, the team selected six of these theories and extracted a series of consciousness metrics from them.
One of these is the global workspace theory, which claims that humans and other animals use a number of specialized systems (also called modules) to perform cognitive tasks such as seeing and hearing. These modules work independently but in parallel and are integrated into a single system to share information.
Long said that 'by looking at the architecture of the system and the way information flows through it', one can assess whether a particular AI system displays the metrics derived from this theory.
Seth was impressed with the team's proposal. He said: 'It is very well thought out, not rhetorical, and clearly states its assumptions. Although I disagree with some of these assumptions, that's perfectly fine, because there's a chance I'm wrong."
The authors say the paper is far from a final conclusion on how to evaluate conscious AI systems, and they hope other researchers will help refine their methodology. But the criteria can already be applied to existing AI systems.
For example, the report evaluates large language models such as ChatGPT and finds that such systems can be said to have some of the indicators of consciousness associated with global workspace theory. However, this work does not imply that any existing AI system is a strong candidate for consciousness, at least not yet.