This post is part of an ongoing series about writing, storytelling, and critique in VR, in partnership with Galatea, a writing and project management tool for immersive stories.
Aaron Koblin has spent the past decade working on some of the most innovative interactive storytelling projects on the planet. In 2014, that trajectory culminated in co-founding Within (known at the time as Vrse) with longtime collaborator Chris Milk. Within made its name early in the VR landscape with standout experiences like Evolution of Verse, The Displaced, and Clouds Over Sidra; pieces that functioned as many viewers’ first taste of immersive storytelling.
This year, Within debuted Life of Us, an effort Koblin led with Chris Milk and Pharrell Williams. Unlike the body of narrative and documentary work the company has produced over the past three years, Life of Us marks a bold leap into interactivity for the company; it’s a multiplayer, interactive experience that takes users in groups of 2-4 people on the journey of life on Earth.
The experience moves through phases of evolution, putting participants in the body of creatures running the gamut from amoeba to post-human cyborg. New environments yield different types of interactivity—as a pterodactyl you fly, as a lizard you race to evade a T-Rex. Most notable about the experience is the group you travel with and the ways you interact with them. Sometimes that means tossing monkeys back and forth. Other times it means hooting and hollering at each other and hearing the crazy sounds created via voice-modulation.
The result is a piece that bridges the gap between a directed narrative experience and a fully interactive social or game environment. It’s something altogether unique to virtual reality (and within the VR landscape itself): a piece that facilitates the catharsis of true agency within the context of a story that’s been crafted by experts.
In the latest episode of Story Unframed, the podcast about immersive stories and the artists behind them, I was able to discuss Life of Us with Koblin, including key insights about creating interactive narratives, such as accounting for user interaction, exercising constraint to facilitate discovery, and considering its evolution as a storytelling language.
User Interaction is Key
At the onset, Koblin and collaborators knew they wanted the piece to incorporate interactivity, so they started testing rough prototypes—and discovered something surprising.
“We started off by doing some multi-player demos where we just had cubes for hands and heads, and we would be able to just talk to each other in a shared space,” said Koblin. “And what we immediately saw was, you don’t actually need photorealism when it’s a real person on the other end, you actually have the humanity pouring out of this abstraction.”
The fact that people tend to be less concerned about something like photorealism is telling with regard to the way interaction will shape immersive narrative in the coming years, but at the end of the day, story is what will carry the piece beyond a passing gimmick.
“How can we use that in a really intentional way to create some kind of narrative experience?” said Koblin. “And for that we immediately went to: what’s the biggest story we could possibly tell? So for Life of Us it was: the entire history of the existence of life on earth in seven and a half minutes.”
Interactive Narrative VR is Still Evolving
This process of rapid prototyping and experimentation based on user interaction is especially important during this moment in time—which Koblin calls VR’s “infancy”—because we are only just beginning to establish what the inherent storytelling grammar of this medium is.
“I think we forget just how much time things like film have existed, and how people know how to watch movies,” said Koblin. “It sounds like kind of a ridiculous thing, but the language of cinema is so well-defined. If there’s a close-up and you see a tear run down someone’s face, you know you’re not up on that person’s face; you didn’t just transform size and scale. There’s a whole vernacular and set of expectations that we have which are just super crystallized and ingrained in who we are. That doesn’t really exist in VR, and quite to the contrary, once you give somebody a little bit of interaction, they kind of want a lot. And they want everything that they do to work and make sense.”
Koblin cites Owlchemy Labs’s Job Simulator as a sort of bellwether for how interactivity invites the creation of narrative by the user.
“It’s incredible because what they do is basically they look at what people are doing, and they go, “’Oh…I have to add that now;’ and then they add that and it’s amazing because everything that you want to do works,” said Koblin. “That gets harder and harder as you start having multiple lands and multiple sequences and all kinds of new environments and changing storylines and scripts. Ultimately I think that’s where it’s going—I think it’s going to fully immersive stories and fully interactive environments.”
All storytelling media are a form of language, and right now, you might imagine us as “speaking VR” at, say, toddler level. So, in designing Life of Us, Koblin explained that Within wanted to strike a precarious balance in making an experience that a novice could pick up with relative ease, while also doing important work in evolving that language.
“Part of what we were trying to do was to set expectations, like, ‘Okay, I get it, I’m on a linear pathway, I get the means of interaction and I understand, here’s a moment and opportunity for me to experiment. There’s a monkey that jumped on my back. I can grab that off, I can hand it to my friend, I can throw it. Let me experiment with the ways that I can interact at this point in time,’” said Koblin. “There’s so many things that you have to take into consideration in terms of, okay, how am I going to move in this fully interactive world? We kind of just took that out of the game, we said, ‘You’re running through the entire history of evolution—like that’s just happening, let it happen. We solved a lot of problems there too; you don’t have to figure out which way you’re going, you don’t have to change your speed, which can be a really jarring experience in VR.”
Sound as Narrative Guide
With a name like Pharrell Williams on the banner, it seems obvious that Life of Us would feature an emphasis on music. And the attention-to-detail in the realm of sound extends even beyond Williams’s work. Working with longtime collaborator McKenzie Stubbert, the team finessed the piece and connected dots through rapid iteration, a process wherein the audio was being tweaked alongside the narrative.
It’s so much about you interacting and talking with the other people and hearing your own voice transform.
“The subconscious and emotional effects of music and sound can’t be underplayed,” said Koblin. “We were trying to figure out…what kind of sound effects [would] make sense and work. [McKenzie] was working on chapters of music, and then we would update him by also putting him through it. He’s kind of a magician so he would just change things as needed. It was a process of feeling it out and experimenting.”
In immersive media, sound—whether it be score, cues, or dialogue—is as much a narrative guide as it is an aesthetic component. If a user hears a sound from their back left, that might be a form of narrative direction. Similarly, building a soundscape that captures the essence of an environment will capture a mood that generates deeper presence. But maybe most importantly for Life of Us, sound plays a huge role in the way people interact with each other (and themselves).
“I think the second you shift your voice—it’s not unlike helium, if you ever give helium to somebody all of a sudden they will say the first thing that comes to mind and act stupid and silly and have fun because they have an excuse; because all of a sudden they’re not themselves anymore, they’re something else, and they have permission to be that way,” said Koblin. “And I think a lot of the music and sound design was about setting a context which would allow you to recognize, ‘Oh my God, this is a little bit ridiculous, and this is a little bit epic, and this is an amazing experience to allow myself to be something completely unlike how I am every other day.’”
Koblin and Milk knew that striking the right balance among all these sounds would be integral to capturing the desired narrative experience.
“It’s so much about you interacting and talking with the other people and hearing your own voice transform,” said Koblin. “So we wanted to make sure neither were drowning each other out—the subconscious effects of music are something Chris and I have been really sensitive to for a long time. Ideally with the music, you don’t oftentimes realize it’s even there but it’s totally guiding your emotional journey subconsciously, or quasi-consciously.”
Constraint Invites Discovery
Through the process of prototyping and editing, Koblin was reminded how valuable a minimalist approach can be in VR storytelling.
A lot of the stuff that I would have expected would be incredible in VR is actually not the most amazing, and some of the more mundane seeming things are.
“I feel like returning to that idea of constrained environments and interactions is just, hugely valuable,” said Koblin. “There’s nothing worse than wanting to do something and not being able to do it. Probably the closest thing worse is FOMO, the constant feeling like, ‘Oh I should have just done that thing and I didn’t get a chance to do it.’ If you can somehow make it feel like the person is super-creative and intelligent for deciding to do the thing you exactly wanted them do, that’s the sweet spot.”
To hit that sweet spot, creators would do well to think about detailed environments rather than plot overload. By focusing on making each moment feel rich in detail, artists give audiences the chance to discover an experience in their own unique way—and feel rewarded for doing so. This is something that often comes as a surprise to creatives taking the leap from film or video games.
“A lot of the stuff that I would have expected would be incredible in VR is actually not the most amazing, and some of the more mundane seeming things are,” said Koblin. “Simple things that are actually more about presence and being in a place work so much better than bombastic roller coaster rides and things that are highly drug-induced experiences. It’s so much about places, and spaces, and about going on a journey; that journey doesn’t have to be 70 miles per hour barrel-rolling.”
Discovery Generates Presence
And in facilitating that sense of discovery, creators guide audiences toward the grail of immersive storytelling: authentic presence.
“The value of discovery is just huge,” said Koblin. “Even just your ability to look around will reveal that you’re in a different mindset.”
Unlike cinema, where audiences are inherently passive observers, in VR, they can participate. When creators foster this agency correctly, they elicit a type of engagement that is rooted as much in life as it is in story.
“If you’re watching a film and something’s being delivered to you, you just sit back and you wait for the thing to happen. When you’re looking around [in VR] all of a sudden you’re an active participant, and you want to feel rewarded for that. You want to feel like, ‘Oh, the fact that I looked up in this Syrian refugee tent and I noticed that the UN logo is actually upside down on top of this tarp.’ All of a sudden that realization is mine and I felt like I was there and I discovered it, and that was meaningful in a different way than a director zooming in on that and showing it to me intentionally. I think that’s part of what we should start playing with and thinking about.”
For Life of Us, Koblin explained that much of the writing process was guided by thinking intentionally about interactivity and user interaction. For that, you have to think beyond what a user will see—you have to account for the myriad ways they are going to want to be and act. You have to intimately understand what you’re trying to accomplish with your work and why. In the case of Life of Us, though lofty in theme, that was to spark joy.
“A lot of this was about excuses,” said Koblin. “It was about giving people excuses to be childlike and have fun again.”
