Oculus Rift Game Inspired by William Gibson's Neuromancer

By Wesley Fenlon

An educational game for the Oculus Rift explores the link between the virtual and the physical.

The Oculus Rift is close, so tantalizingly close, to fulfilling the potential it was born for: Depicting the virtual reality of William Gibson's cyberspace. For a MIT Media Lab class, teacher and maker Greg Borenstein recently built an educational game prototype for the Rift called Case and Molly, named after the protagonists of Gibson's Neuromancer. Case is the pro hacker or cyber jockey, who feels more at home in the information matrix than in the real world. Molly is the expert bodyguard and assassin, her body enhanced to be faster, more lethal. Borenstein's game is "about the coordination between the virtual and the physical, between 'cyberspace' and 'meat'."

When Case and Molly work together to perform a heist in Neuromancer, Case can "ride along" by tapping into a neural feed from a chip in Molly's head. He can watch, but can only communicate with her via a binary bit of information. When he's not in Molly's head, Case is hacking through the matrix. Borenstein compares this duality with everyday life today--the way we stare at our phone screens as we walk down the street, drive and watch a GPS map, write email and glance at a Twitter feed. " 'Case and Molly' uses the mechanics and aesthetics of Neuromancer’s account of cyberspace/meatspace coordination to explore this dynamic," he writes.

But the most interesting element of the experiment is replicating that sensation of "riding along" with the Oculus Rift--of knowing what it's like to see through someone else's eyes. As a game, it's incredibly simple. Molly wears a pair of iPhones, duct taped together, that provide a 3D video feed for Case. She has 30 seconds to make her way to a specific room under Case's guidance. He sees instructions for her destination laid over the video, but can only communicate by flashing a green/red light on Molly's heads up display, which is a third iPhone. If she makes it into a room, Case then switches over to cyberspace, a simple puzzle that requires him to arrange a series of cubes based on how many times they blink.

Meatspace, via the Oculus Rift, had messed with his mind.

When Borenstein, playing as Case, made it to the cyberspace puzzle portion of the game--a game he designed--he found he couldn't do it. Meatspace, via the Oculus Rift, had messed with his mind. He writes:

"The virtual reality interface and the rapid switches to and from Molly’s POV left me cognitively overwhelmed. The first time we successfully completed a Molly turn, I found I couldn’t solve the puzzle because I’d essentially lost the ability to count. Even though I’d designed the puzzle and played it dozens of times in the course of implementing it, I failed because I couldn’t stay focused enough to retain the number of blinks of each cube and where they should fit in the sorting. This effect was way worse than the common distractions of email, Twitter, texts, and IM many of us live with in today’s real computing environments."

Borenstein's description of the sensory ride along is my favorite bit.

Photo credit: Greg Borenstein

This is the kind of experience that could take the Oculus Rift beyond being a cool way to play video games and into something that affects how we experience the world:

"One of the strongest resonances [with Gibson's writing] was the dissonance between the virtual reality experience and being thrust into someone else’s point of view. In Neuromancer, Gibson describes Case’s first experience of 'switching' into Molly’s subjective experience, as broadcast by a newly installed 'SimStim' unit:

The abrupt jolt into other flesh. Matrix gone, a wave of sound and color…For a few frightened seconds he fought helplessly to control her body. Then he willed himself into passivity, became the passenger behind her eyes.

"This dual description of sensory richness and panicked helplessness closely matches what it feels like to see someone else’s point of view in 3D. In Molly mode, the game takes the view from each of two iPhones aligned into a stereo pair and streams them into each eye of the the Oculus Rift. The resulting 3D illusion is surprisingly effective. When I first got it working, I had a lab mate carry the pair of iPhones around, placing me into different points of view. I found myself gripping the arms of my chair, white-knuckled as he flew the camera over furniture and through obstacles around the room. In conventional VR applications, the Oculus works by head tracking, making the motions of your head control the direction of a pair of cameras within the virtual scene. Losing that control, having your head turned for you, and having your actual head movements do nothing is extremely disorienting."

Photo credit: Greg Borenstein

Case and Molly is an educational project, which means it will never become a commercial game for the Rift--its code is available to download, though. But it seems like it presents two powerful ideas for future Oculus products. One would be the pure game, a true 3D representation of Gibson's cyberspace. It's been nearly 30 years, and no movie or video game has matched the evocative visuals of Gibson's matrix as they're described on the page. That's a game world I could get lost in.

The other option, of course, is the sensory experience--riding along in someone else's head, seeing what they see, but being completely helpless to control anything. Based on Borenstein's description, it would probably end up even more disorienting than a 3D simulation of cyberspace.