Why Virtual Reality Can't Totally Fool the Brain

By Wesley Fenlon

Resolution and refresh rates aren't the entire problem--the way our brain understands its surroundings isn't quite fooled by virtual reality.

Combining the Oculus Rift with an omni-directional treadmill is the closest approximation of the Holodeck the technology community has been able to whip up so far. It's still not quite so immersive as Star Trek: The Next Generation's impossibly perfect simulator, but it's not a bad first step. But it's hard to say if we'll ever reach that goal. As Valve discussed at this year's Game Developers Conference, virtual reality technology still has a long way to go on technical challenges like latency and resolution. And a recent study from io9 highlights an entirely different challenge that stands between VR and the holodeck: thoroughly fooling the brain's sense of "place."

The study focused on the brain's hippocampus, which includes cells referred to as "place cells." Another study io9 referenced explains their importance: "More than three decades of research have demonstrated a role for hippocampal place cells in representation of the spatial environment in the brain. New studies have shown that place cells are part of a broader circuit for dynamic representation of self-location...place cells and grid cells may form the basis for quantitative spatiotemporal representation of places, routes, and associated experiences during behavior and in memory."

To test out how the brain deals with its spatial environment, scientists started with rats in a maze. When rats are first dropped into a maze, their place cells light up as they map out their new surroundings. The rats seemed to do this quickly and easily, so the scientists conducted an experiment: Could the rats just as easily map their surroundings in a virtual environment? Writes io9:

For the study, the researchers tried to create two apparently identical worlds, one real (RW) and one virtual (VR). Each environment consisted of a linear track in the center of a square room with distinct visual cues on each of the four walls. These cues were nearly identical in both environments, but the rats' bodies were fixed in VR — thus minimizing (or even eliminating) other important spatial cues, like balance. So, the only incoming environmental data during VR exposure were the visual cues and self-motion.

The study's results showed a big gap between how rats perceived the real world and the virtual world. Here are a couple excerpts that are pretty easy to understand:

"In VR, place cells showed robust spatial selectivity; however, only 20% were track active, compared with 45% in the RW...In addition, bidirectional cells preferentially encoded distance along the track in VR, while encoding absolute position in RW...These results reveal cooperative and competitive interactions between sensory cues for control over hippocampal spatiotemporal selectivity and theta rhythm."

Essentially, virtual reality can't convince the brain of its position; cognitive mapping uses other clues to fully map out environmental awareness. But this isn't necessarily a roadblock to making virtual reality fun and partially immersive--it just means we're not going to be perfectly replicating the indistinguishable-from-reality experience of the Holodeck. We can probably live with that limitation for a few decades.