How can we elicit emotion in immersive technology?

The objective of immersive technology is to create unforgettable experiences. The underlying ambition when an individual interacts with augmented and virtual reality (AR/ VR), is ultimately to build an enhanced, engaging experience, that surpasses non-interactive alternatives.

To truly understand how to cultivate this, firstly we have to understand emotion and, more importantly, how to elicit emotive responses. The REVEAL Research Centre based at the University of Bath has been studying just this to aid the immersive technology community, most specifically immersive designers. 

We caught up with Dr. Christof Lutteroth, Director of REVEAL, to learn more about his research and how it will impact the future of VR. The team’s focus is to bring different areas of expertise together to literally augment the quality of research and therefore impact of the outcomes. REVEAL consists of researchers interested in AR, VR, and XR, as well as those working on making the digital more physical with haptic technology, expanding into facilitating new ways of touch and interaction. This research is conducted in tandem with specialist tech companies, such as Ultraleap and Zubr, our Immersive Month sponsors, to further substantiate the findings.

Primarily, the REVEAL team works on creating simulations for training and planning. For example, one of their recent papers looked at street harassment to investigate if immersive tech can help address the problem. In their research, the team designs experiences that mostly haven’t been designed in VR before and then analyses them to see what makes the experience come to life. Their results from the street harassment project subsequently deepened the understanding of the design decisions that contribute to a realistic psychological experience.

Christof explains that one thing he is particularly passionate about is measuring emotion: “Very often our entertainment experiences and training scenarios rely on eliciting emotions. So, for entertainment, what keeps us glued to the screen or to our headset is not just a mental experience, it’s an emotional experience. However, designing emotional experiences, especially in VR, is really hard.

“When you’ve got a screen, it’s much easier to direct people’s attention, you’d basically just show them exactly what you want them to see. However, when you switch over to a medium where the user has a lot more control, such as virtual reality, it becomes a whole different ballgame.”

“If you want your audience to be afraid and you’ve got the real-time sensor data and information, you could build that into your system without any guesswork.”

Naturally, with a 360 experience, the user might not be looking at what you want them to see, so you have to really attract people’s attention and take this into consideration as you’re guiding people through the experience. Christof continues, “In the street harassment simulation, we found that giving people some agency helped with this. For example, allowing them to push back or to say something. Allowing participants to interact in the simulation brought it onto a new level really, which made it much more effective and therefore elicited more emotions. People felt a lot more present in the experience.”

The main objective behind these studies is to obtain a position where trial and error is reduced, or ideally completely eradicated. Christof elaborates, “Very often, things simply don’t work the way you expect them. We’re trying to develop the technology that helps designers be more systematic and, in the end, more successful in designing the experiences.”

Within VR, there is a plethora of technical parameters you can change – from investing in quality headsets to incorporating a wider field of view, the list goes on. In their studies, REVEAL can vary some of these aspects, all with the mission to “model what really makes the experience tick,” explains Christof.

The prospect of creating experiences that responds to people’s emotions is particularly groundbreaking, but more extensive research into how this works with interactive experiences is required before we can reach this goal. REVEAL’s work is actually looking at the real-time to quantify, how does your audience feel?

“We don’t just want to do ivory tower science; we want to bring our work out into the real world.”

In the long run, this research will make design easier. Christof explains, “It will reduce the number of design cycles that people have to go through in order to create high-quality experiences because they can find out much more directly what their design is actually doing.

“If you want your audience to be afraid and you’ve got the real-time sensor data and information, you could build that into your system without any guesswork. It will enable experiences to be emotionally intelligent, you could say. You could create something that knows how the user is feeling, meaning you could create art that really interacts on an emotional level with people, literally interacting with their emotions.”

On top of this, REVEAL is also looking at interaction technology to improve our experience within virtual worlds. Christof tells us, “In the future, people will spend more and more time in AR and VR with the growth of the metaverse. One of the challenges has always been, how do I quickly enter text?”

To resolve this problem, REVEAL has been building technology that uses eye gaze to help improve efficiency, whilst also bolstering the flexibility of using immersive technologies. Christof gave us the example of someone wearing an AR headset on a bus. Using this state-of-the-art tech, users can easily connect with others, without disengaging from the immersive tech. It works using a simple concept – tapping. Christof elaborates, “So just imagine, instead of typing, you could just tap on your thighs, or just on any surface that you’ve got. Our eye gaze technology is able to figure out what you’re typing to help you put together sentences very, very quickly.”

Using AR glasses, the idea is to tap in the same way you would usually type. Pair this with some clever eye gaze interaction technology, and research has shown that typing speed can be improved by 70% – whether or not participants were previously fast or slow typers. 

REVEAL also evaluates experiences that tech companies, such as Zubr, have built to validate their credibility. Christof explains further, “If you want to sell something, for example, you’ve got some training simulation, or you’ve got a simulation that is supposed to put the user in a particular mindset, then we can actually tell you whether it really does that or not. We’ve got the expertise to run studies and collect empirical data to substantiate this.

“It’s interesting for us to do that kind of work. Often we have to look for a vehicle to do that, so usually, we will look for research funding together. For example, we go to UKRI or Innovate UK, and we apply together with industry partners. It’s a powerful way of doing work together.”

So if you’ve created an immersive experience that you’d like to really put to the test and improve, reach out to Christof and the REVEAL team, who are keen to open the door to new opportunities.