Scientists probe the nature of the mind

What is the mind? For thousands of years, philosophers, religious figures and scientists have tried to answer this question, but so far, the nature of the mind has eluded understanding. In the field of cognitive science, one idea that gained traction in the mid-20th century is the computational theory of mind, which posits that the mind acts as an isolated computer, with the brain receiving and processing inputs from the environment and producing specific actions and interactions as outputs.

But Professor Tom Froese, who leads the newly formed Embodied Cognitive Science Unit at the Okinawa Institute of Science and Technology Graduate University (OIST), is challenging this hypothesis. His project team is exploring the concept of embodied cognition, where the mind is not only based on the brain, but also extends into the body and the environment.

“In embodied cognition, interactions matter,” said Prof. Froese. “Instead of just being a product of the mind, interactions are a fundamental part of the mind, occurring between the brain, the body and other agents in the environment.”

Professor Tom Froese, who leads the Embodied Cognitive Science Unit at OIST.

The value of interactions in embodied cognition is what initially drew Prof. Froese towards this theory.

He argues that if cognition is not embodied and the mind is limited to what happens in the brain, this means that an individual’s entire life, full of social interactions, can simply be reduced to neural activity. When taken to the extreme in thought experiments, all of human experience could instead be achieved by an advanced supercomputer feeding simulated inputs to a brain suspended in a vat.

“For me, this is a deeply unpleasant implication; I think real-life social interactions matter immensely,” said Prof. Froese. “Currently, no-one has been able to experimentally prove that this is how the mind works. So, in this neutral space, we might as well try and test the more satisfying theory of embodied cognition instead.”

The importance of agency

One piece of evidence that could support embodied cognition is whether active engagement – or agency – helps the mind to learn new things, such as the ability to perceive the environment.

Here at OIST, the Embodied Cognitive Science Unit are planning to test this with a novel setup involving a huge robotic arm and a little sensory substitution device, which will permit them to record the hand movements and tactile sensations of participants actively engaged in a perceptual task, and then replay those same movements and sensations to participants passively undergoing the task.

On the left, Prof. Froese showcases the robotic arm, that can track and replay arm movements. On the right, the handheld sensory substitution device, which allows participants to “see” obstacles using vibrations.

“This will be a first, in terms of creating an experiment that accurately recreates the task without agency, by keeping the exact relationship between how the arm moves and what it senses,” said Prof. Froese. “If the individuals without agency perceive less well, this pinpoints that active engagement – something that can’t be achieved by a disembodied brain in a vat – is important for total cognition, therefore providing support for the idea of embodied cognition.”

Diagnosing mental disorders as interaction disorders

The challenge to experimentally prove how the mind works may seem far removed from our day-to-day lives, but in fact, it has tangible real-world implications, including the diagnosis of mental diseases, such as schizophrenia.

“Traditionally, schizophrenia was defined very much by its symptoms, such as hallucinations, and catatonic behavior. But now it's much more thought of as rooted in problems with social interactions, where the naturalness with which people interact with others breaks down,” Prof. Froese explained. “In fact, some individuals with schizophrenia report a sense of disembodiment, where their bodies feel like robots that simply receive information and are controlled from the outside.

“From these reports, our unit believes that the that mainstream thinking of cognition as an isolated computer may not only be wrong, but actually describes a pathological state of being.”

To gain more information about how people with schizophrenia interact, Prof. Froese is collaborating with Heidelberg University, Germany. There, researchers are testing people with schizophrenia using an experiment known as the “perceptual crossing paradigm”, which was adapted by Prof. Froese.

The perceptual crossing paradigm is a virtual reality game which determines how well humans can sense when they are engaged in a genuine interaction with another human.

Prof. Froese demonstrates the perceptual crossing paradigm – a virtual reality game where participants rely on vibrations to interact with static and moving objects, as well as another player’s virtual avatar.

In a previous experiment, Prof. Froese found that people could distinguish the other player’s movements from that of the other virtual objects, showing that they were sensitive to the mutuality of interacting with the other player, rather than simply the presence or the motion of other objects. The participants also tended to recognize each other at almost the same time, only around a few seconds apart.

“These results suggest that there may be some large-scale neural integration connecting the two players’ minds,” said Prof. Froese. “Now, we’re excited to see if we get different results in people with schizophrenia, which could provide further insight into the disorder.”

With this aim, Prof. Froese is collaborating with researchers at KU Leuven in conducting a long-term study of individuals who participated in the perceptual crossing experiments as adolescents. They plan to follow up on them to see if any of them later develop mental disorders.

“We hope that with the benefit of hindsight, we may be able to see subtle differences in the original interaction data set, which could then be used as an early detection system,” said Prof. Froese.

The death of AI

Although the theory of embodied cognition has profound positive implications for society, for Prof. Froese there is one personal downside – an end to the dream of replicating the human mind in AI.

“I originally wanted to create better and more intelligent AI for games but I soon realized we didn’t yet properly understand the mind – that’s how I ended up switching to the field of cognitive science,” he said. “And now, with an understanding of embodied cognition, I feel like life is an essential and fundamental element needed for real understanding.”

Unlike an inanimate object like a robot, a living body must constantly be doing things in order to survive – its existence is precarious and constantly subjected to potential annihilation. Prof. Froese believes this constant struggle for existence is required to develop a mind that cares about things, and which therefore is capable of developing thought and understanding.

“Robots and AI are very good at mimicking understanding, but this is simply an illusion – they are easily fooled and unable to adapt beyond a rigid set of pre-defined rules,” said Prof. Froese. “Without life, which brings with it the possibilities of success and failure, there is no meaning. Therefore, AI that actually understands like we do may be a pipedream.”

Instead, Prof. Froese believes the future lies with human-computer interfaces, where AI can be used to empower the human mind, rather than to replicate it.

“If you look at the really transformative technologies that've changed our world, it hasn’t occurred by trying to make AI function autonomously, it's with inventions such as the internet, personal computers and smartphones,” said Prof. Froese. “In other words, technologies that revolutionize how we interact with each other.”