This summer, the Sundance Film Festival hosted the premiere of Notes on Blindness. It's a film about John Hull, a theologian who spent 16 years chronicling his degenerative blindness in an audio journal before total darkness fell in 1983.
Alongside the film, Agat Films/Ex Nihilo and Audiogaming released "Notes on Blindness: Into Darkness," an immersive VR project that builds on that audio odyssey, and both supplements and promotes the original work. Funded by ARTE, the studios used binaural audio and real-time 3D animation to give people the sense of going blind alongside Hull—a neat juxtaposition to how the National MS Society used VR to help MS patients "relive" certain passions.
Each scene addresses a memory and location from Hull's audio diary, and sound is used to create visual cues that build on the feeling that other senses are heightening—even visually compensating—as your eyes dim.
The project won the Storyscapes Award at the Tribeca Film Festival and the Alternate Realities VR Award at Sheffield Doc/Fest. It took two and a half years to perfect across multiple platforms. In a panel at MIPCOM last week (which I moderated), new media lead Arnaud Colinart of Agat/Ex Nihilo and CEO Amaury La Burthe of Audiogaming explained how they brought it to life, and what they took from the gaming world.
"We started from the audio material and the story John wanted to tell," La Burthe said. "We isolated several moments in the tapes… and tried to elaborate interactive situations."
One of the hardest aspects of the project was the very constraint that made it interesting. "We realized that by doing something that was only audio-based, we were losing a lot of people. We were losing the sighted audience," La Burthe went on. This led to creating visuals that can be "seen" only when sound is emitted in the transcripts.
In a VR experience, people impose their own timing. Some will rush through it, while others linger—making chronology difficult. (Director Alex Smith recently likened VR shooting to directing theater, in the sense that it's a living, unique experience for everyone, with nowhere for a cameraman or missed shot to hide.)
To compensate for this, they drew from a gaming principle called Flow Theory—the notion that a game can't be too simple (thus boring) or too complex (anxiety-inducing).
"Depending on how much you look around, the story's going to be slower," La Burthe said. "If you look around a lot, we're going to trigger a few additional things."
Who can get into an experience this complex? Colinart admitted it was "clearly a niche," but they aimed for festivals around the world to bait the tech and artistic communities. Festivals served as testbeds for future updates. The pair noted moments when, for example, people began to cry. It also appeared on the content homepage of the Samsung Gear VR headset.
Tech complications abounded, though. Because VR platforms aren't agnostic, the project had to be rebuilt for each one. "Forty-eight hours before the release, we discovered that the new version of iOS 10 introduced a major bug!" La Burthe revealed. "You don't have technical problems like that on an animated series. You're not going to lose all your rendering 48 hours before showing it to the public."
But the pair saw the project as a powerful lesson in empathy, for themselves and others. Offstage, La Burthe explained that virtual reality can help people feel what others experience.
Colinart also sees it as a teaching opportunity for entertainment at large.
"The success of studios such as Pixar and big video game studios is based on the possibility to fail and learn from these mistakes," he said. "In the culture of TV, especially in Europe, we don't have this culture of failure. We need to succeed. And it's impossible to start these kinds of projects if you are not ready to fail, learn from your mistakes, and restart."
The Notes on Blindness VR experience is available on the Samsung Gear, and via mobile or Cardboard for both iOS and Android. The studios also created a supplementary short film called Radio H., based on the reflections of John Hull's daughter, Imogen Hull, who, as a child, created an audio diary at the same time he did:
