Frontiers Science Hero: Mel Slater from Frontiers on Vimeo.

Prof. Mel Slater has always been intrigued by the science that lies behind virtual reality. “I think one of the things that fascinated me about it was the question why people, when they go into a virtual reality even though what they see may not look real, they nevertheless have a strong illusion that what is happening is real and also a strong illusion that they are in the virtual place,” he said.

Mel Slater is an ICREA Research Professor at the University of Barcelona, and became Professor of Virtual Environments at University College London in 1997.

Slater started working in the area of virtual reality during the early 1990s with the aim of understanding our mindset when it comes to a virtual world. “I set out to try to understand why this illusion happens and what factors we as engineers can introduce into our applications to promote those feelings and realistic responses to virtual reality,” he said.

The power of virtual reality

Throughout his career, Slater has made several important discoveries. “I was one of the first people in this field who worked in this area of understanding the impact of virtual reality on people’s perception and behavior. Our group was a pioneer in exploiting virtual reality for the purposes of understanding body representation – how the brain represents the body,” he stated.

Virtual reality creates a digital environment which stimulates the physical sense of presence. Slater elaborated says that when you are in a virtual reality, everything you see is virtual. Even when you look down towards yourself or in a virtual mirror and see a body, that body is virtual.  He says everything can be linked back to the brain. “We discovered the brain very quickly comes to accept — or generates the perceptual illusion — that the virtual body the person sees is his or her own body. This opened up a whole new branch of research where virtual reality was used in the understanding of body representation,” he said.

Over the years, his group conducted studies at the Queen Mary University of London (formerly Queen Mary and Westfield College), UCL and now University of Barcelona. He says that the work in each of these institutes contributed to the research.

Are ‘Fifa’ and ‘Call of Duty’ part of virtual reality?

When thinking about virtual reality, we immediately associate it with video and computer games. However Slater identified that until actually very recently there has been a big difference between the two. “In virtual reality, you are surrounded with a life-sized virtual world where you interact with virtual characters that are the same size as you or who come into your space. They can look you in the eyes or touch you,” he said. “It really is a different experience to playing video games where you are looking at an external screen, where you interact with virtual characters much smaller than you.”

The difference between virtual reality and video games is that in virtual reality your entire body is engaged in a life-sized ongoing series of events which you see in stereo 3D where, as you move your head or your eyes, the world is updated accordingly. “In scientific research we use virtual reality to understand some aspects of perception, in particular the illusion of being in a place. In virtual reality we can study that because one of the whole points of virtual reality is that it can be used to change the place of where you are. Everything you see and hear is generated by a computer,” Slater explained.

The main thing that is of importance in order to understand the connection between virtual reality and video games is the understanding of body representation, the field of research Slater’s team focuses on. He elaborated this with the help of an example. “If I were wearing a head mounted display now and in virtual reality looked down towards myself, I would see a virtual body rather than my real one. Also, through real time motion capture, I can, when I move my real body, see my virtual body move similarly.”

In the recent past there has been an increasing convergence between virtual reality and video games. There have been pretty high quality, low cost consumer, virtual reality systems on the market in particular for head mounted displays which makes it affordable for people to use in their home. “As an example, most of our experiments in the past have been done on systems where the cost ranged between one hundred thousand and half a million euros. Now we can get pretty good set ups including a computer for under two thousand euros. Thus, the technology is becoming available in homes. There are many companies which have realized this and they are beginning to produce computer games that will work in virtual reality,” Slater stated.

New technology opens up new dangers

We have to think beyond just video games. Virtual reality is used scientifically to display bigger than life sized scientific objects such as molecules or pieces of brain, where the researcher can walk around them and see them from different perspectives.

The reason for this rapid development of application in virtual reality is due to the advancement in technology. “In the early 1990s, the kind of environment you could create was very blocky looking with an old style computer graphics where a whole human body might be represented by 100 polygons. Nowadays the level of computer graphics has improved to such an extent that getting high level realism is no longer a problem at all,” Slater said.

Slater further added that this technology not only opened new doors but dangers as well as virtual reality mobilizes your entire body and you are in this life size virtual world. “I think that shoot up games in this kind of environment are really different from the experimental point of view compared with shoot up games where you are interacting with little characters on the screen. It introduced new problems both ethical and practical problems which the industry and the researchers need to address,” he said.

How real does it need to be? 

The brain is connected to our perceptions of our surroundings. More interestingly scientifically and theoretically is that the level of realism does not make a huge difference. “The level of realism is still an open question. There are some areas where I think it is very important, but what we ourselves are studying is where the level of realism does not seem to impact your illusion of being in the virtual world, but it may impact the extent to which you take events as real,” Slater said.

This idea has been illustrated with the help of a dilemma experiment. “We have done experiments where people are involved in a moral dilemma, the kind where in order to save five people you may end up sacrificing one,” he said. “Something about the brain therefore tells us the virtual figure looks like a human and behaves like a human and the safest thing for me to do is to act like it is a human. By the time a high level of cognitive function kicks in that it is not real it is too late and you already responded.”

Our brain automatically reacts and adjusts to the situations it finds itself in. He says there are some aspects of perception where the brain fills in the details as the person looks around. The process happens quickly, and while there are some higher cognitive processes that slow down the reaction, by then it is usually already too late as you are already sweating or your heart rate has already increased by several beats per minute.

Slater and his team of researchers recently completed a project called “The Immortality project”. This was connected to research in social psychology which looked at the impact of people’s knowledge of their ultimate mortality on their life, attitudes and behaviors.

“This project was about giving people a personal experience of a life and death cycle and then seeing how that affects certain attitudes and believes that they hold,” Slater said.

The papers on this project will soon be published. It is being exhibited at the Human+ Exhibition at the Centre for Contemporary Culture in Barcelona (CCCB, Human+, http://www.cccb.org/en/exhibitions/file/human-/129032) between October 2015 and April 2016.

His work has received coverage from Financial Times Magazine, Popular Science, MedicalXpress,  The New Yorker, Yahoo News, BBC [eg http://www.bbc.com/news/science-environment-23709836%5D

Mel Slater’s YouTube Channel

Recent published papers by Mel Slater:

Over my fake body: body ownership illusions for studying the multisensory basis of own-body perception: http://journal.frontiersin.org/article/10.3389/fnhum.2015.00141/abstract

Virtual character personality influences participant attitudes and behavior – an interview with a virtual human character about her social anxiety: http://journal.frontiersin.org/article/10.3389/frobt.2015.00001/abstract

Violating body movement semantics: Neural signatures of self-generated and external-generated errors: http://www.sciencedirect.com/science/article/pii/S1053811915007314

Conversations between self and self as Sigmund Freud-A virtual body ownership paradigm for self counselling. http://www.nature.com/articles/srep13899