im/material

Im/material Event #3: VR, Immersion, and Narrative

Photo Credit to Arcadia Festival by Biome Collective; Photography by Erika Stevenson; Game by Lynn Parker of Abertay University.

The third im/material network event, VR, immersion, and narrative, was held at the University of Glasgow on 4 October 2019. The discussion sought to bring out how immersive technologies have created new challenges in storytelling, especially in video games, where the player wrests even more control from the game designers. Our four panel members are specialists in narrative design in virtual and augmented reality: Jon King (Senior Designer from PlayStation, London Studio), Rhoda Ellis (PhD Candidate, DJCAD, University of Dundee), Chris McLaughlin (Tech Director of Moon Mode), and Mal Abbas (Co-Founder of Biome Collective). The discussion was moderated by Matthew Barr (Lecturer at the University of Glasgow).

What VR project are you particularly proud of or is there one that you particularly admire?

Jon King said that he was particularly proud of the work he’s done in VR Worlds, especially Ocean’s Descent, because building a VR narrative required him to unlearn things from making traditional games and to experiment. He’s particularly proud of the powerful experience Ocean’s Descent produced.

Rhoda Ellis found the early VR work of Char Davies inspiring because in her work the player controlled the experiences using their physical bodies. Rhoda is proud of her degree show project ‘Being-in-the-Gallery’, which also melded the physical with VR – in both VR and reality, the user entered the gallery space and was able interact physically with the sculpture they viewed digitally.

Malath Abbas agreed with Jon that Biome Collective’s first VR Project, Garden, also required experimentation and unlearning to create the narrative. By focusing on the user experience that they wanted, they were able to build an engaging experience with simplified graphics. He was inspired by a visit to the Emblematic group three years ago, where he was able to see some of their early prototypes. One documentary immersive experience addressed domestic violence; the use of very basic 3D visualisation in a spatial environment combined with real audio created a very powerful experience.

Chris McLaughlin was most proud of a two-player VR game that Moon Mode have not found a release route for yet, Tiny Escape. One person plays a tiny alien trying to escape from a giant robot – the best reactions the game received were from children and their parents when children play as the giant robot because of the role reversal from reality to the virtual. Chris admired the narrative construction of Virtual Virtual Reality because it provides so much choice for the player while still maintaining their storyline.

How do you tell stories in VR experiences – is it different from other games or other forms of storytelling?

dronepicr [CC BY (https://creativecommons.org/licenses/by/2.0)]

Three of our panel members are game designers, and each of them stated that there were certain elements of narrative that would work in a traditional video game, but they needed to unlearn or experiment with in developing a VR game. Environmental storytelling becomes much more powerful in VR, and audio becomes much more integral to storytelling, whether through the story’s characters or a television or radio in the environment. The timing of cues and narrative notes are key to guiding the player through the main story.

The actions and performance of the in-game characters can also contribute to storytelling, especially to build an emotional connection to the characters and to create immersion. It’s easy for players to lose that immersion if the characters do not acknowledge the player or react appropriately to players acting in unpredictable ways.

Malath argued that there is a certain threshold that a VR experience has to achieve to get the players to buy into the experience and suspend their disbelief; once that threshold is achieved, you can get more creative while still maintaining that immersion.

To build these more believable and immersive environments and characters, Rhoda suggested that adopting approaches to narrative from theatre or other media, rather than those approaches used in traditional video games, can enhance these experiences further.

Are there any unique opportunities in immersive storytelling in VR?

Virtual reality opens up a new level of immersion that can elevate feelings and emotions in the player. In other media, like film, there is a divide between you and the people you are watching. By building an experience beyond the visual, including sound, movement and touch, it becomes easier to get ‘you’ to care about who or what is in front of you and to accept the rules of the game. After you get beyond the threshold of disbelief, the characters are talking to ‘you’. Because it is ‘you’ that characters and the environment are reacting to, the experience becomes ‘your’ unique experience. Achievements in the game become ‘your’ achievements, and ‘your’ memories, and this is powerful. The challenge now is to shift this from ‘I, me’-centred experience to a social experience that can be shared with others.

Why is getting immersed in a first-person book different from being immersed in a VR narrative?

Photo Credit to Arcadia Festival by Biome Collective; Photography by Erika Stevenson; Game by Lynn Parker of Abertay University.

A well-written book does achieve a particular level of depth that will take you on a journey, and you become immersed by imagining voices for characters and imagining sounds that are described in the text. But a book is usually an authored linear experience, encountered page by page. Virtual Reality is different in that, while there is an overarching authored experience, there is another layer of immersion where the player can create their own experience or their own game. You can go do something else and create your own story while the larger controlled story is happening around you, whether it is playing with the items in the glovebox of a car or playing basketball with paper balls and a bin.

There’s currently a lot of interest in using VR for educational experiences, but does it exclude people? How accessible is VR?

Many VR experiences are not very accessible at the moment, and it is something that the panel members agreed needs to be thought about more. Most VR experiences do not take into account those individuals who have trouble reading text, or those who have limited movement in head, neck or hands. Rhoda in particular felt strongly about this, as her latest piece involves creating a VR experience of a sculpture park for wheelchair users; she found it was necessary to build in mechanisms to move in VR for people with these restrictions and using only one button. VR and other immersive technologies have the opportunity to tell new stories in interesting, empathetic ways. They hoped that this will be brought to the forefront sooner rather than later, because this is one of the big issues facing VR right now, but it is more likely to do so once the technology is offered on the mass market and once game makers and artists can invest in making more diverse content.

Looking to the future, what do you think is next for VR? What are the opportunities for more stories as the technology improves?

Photo Credit to Arcadia Festival by Biome Collective; Photography by Erika Stevenson.

Each member of the panel agreed that VR will likely move towards telling more stories from multiple perspectives and sharing VR experiences between multiple players. Jon described a game called ‘Trover Saves the Universe’, where one that plays with multiple perspectives, where it’s a first person game being controlled from a third-person perspective. He argued that VR allows game designers to think differently about perspective and that this will be increasingly explored in storytelling. The others agreed that more creative, experimental approaches to storytelling will become more prevalent when VR appeals to a mass-market, but all were excited by the prospect of getting multiple players into the same VR experience to tell powerful stories.

The subsequent discussion with the audience highlighted several important issues that need to be addressed before we create massive shared VR experiences, including:

What will the role of linguistic diversity and the representation of language be in VR and other immersive experiences?

Is there a difference between what is a ‘game’ and what is an ‘experience’? Is there room for both plot-based VR and entirely exploration-based VR?

Is it possible to lose your perception of reality once you’ve reached the ‘threshold of disbelief’?

While there are bodies that regulate what types of narratives are told in other types of media (to combat lies, deep fakes, etc told in the context of television), but there isn’t a recourse to dispute problematic narratives in VR. Does the industry see a need for a centralised body to address these issues of ethics?

Because these mixed reality technologies can be used to develop empathetic experiences, how feasible would it be to bring these approaches into the health care settings of well-being and counselling to help cope with certain traumas?

Games historically haven’t treated women particularly well, especially in online social spaces, so this is something that needs to be addressed before creating VR experiences that offer shared experiences between multiple players – is there a way to combat negative online behaviour in VR? Or is it up to the user to be self-selecting about the spaces they inhabit in VR?

Has VR changed the role of the environment artist in the production process?

Does VR necessitate building a whole world rather than just building a game more so than other video games? Does a VR experience need to be considered a game, or is it just a tool?

If we can do anything in VR, can it be used as an empathy machine?