Hands-On with Avegant's Virtual Retinal Display Prototype

By Norman Chan

A new head-mounted display that uses DLP micro-mirror technology to project and reflect high-resolution images into your eyes. The result is a stereoscopic image with almost no pixelation and an incredibly high fill factor--meaning no screen door effect. In fact, you can't even see the pixels because there is no actual screen.

I was admittedly a bit skeptical when Grant Martin of startup Avegant reached out to me about a new pair of head-mounted displays the startup has been developing for living room entertainment. We've tested other wearable displays that want to replace your big-screen TV, like Sony's HMZ-T1P and Zeiss's Cinemizer, but have never been impressed by the quality of these OLED glasses, at least not enough to recommend them as an alternative to a flatscreen panel for movie viewing. The Oculus Rift made strides in getting us comfortable with the idea of using HMDs with its low-latency head-tracking and use of lenses for immersive gaming, but anyone who watched our Octoberkast 24-hour live stream knows that it still needs to overcome some technical hurdles to be fit for extended use. Display resolution and image fidelity isn't the problem of HMDs, it's eye strain.

So when Grant and Avegant software lead Yobie Benjamin set up their company's prototype "Virtual Retinal Display"--don't worry, that's not the final product's name--in our offices today to demo some video clips and a PlayStation game, they had to preface it with an in-depth explanation as to why their Avegant's glasses technology is completely different than what we've seen before. The first thing I had to understand, said Yobie, was how the human eye works. The pupil, at the front of the eye, is like the aperture on a camera--the hole that light travels through. That hole grows and shrinks using the iris muscle, restricting or letting more light in depending on the intensity of the light source. But what really matters is the retina--the back of the eye that actually receives light and converts it into signals for your brain. Getting the right amount of light to the retina is what Avegant cares about.

The second lesson was another in human biology, continued Yobie, and understanding that the vast majority of light that our retinas receive is reflected light. It's light that has bounced from the sun or bulbs off many surfaces before it gets to our eyes--so human evolution has conditioned our our eyes to be more comfortable with reflected light. Eye strain occurs when we're staring at directly emitted light, like that coming from a phone display or computer monitor. Or, as you can surmise, every head-mounted display on the market so far. That's the problem Avegant thinks it has solved with its technology.

Instead of using LCD or OLED display, Avegant's Virtual Retinal Display actually has no displays at all. What it has are two RGB LEDs (one for each eye) that each emit a controlled amount of light at a chip with millions of microscopically small mirrors, bouncing the right colors directly into your retina. In effect, it's a low-power projector that's shining reflected light into your eyes. If micromirror-based image projection sounds familiar, that's because it's the same technology used by old DLP televisions and current DMD projectors. And indeed, the mirror matrix in these glasses are Texas Instrument DLP chips, adapted using proprietary technology for HMD use.

The result is a stereoscopic image with almost no pixelation and an incredibly high fill factor--meaning no screen door effect. In fact, you can't even see the pixels because there is no actual screen. You just see one fused image projected through an optical lens in front of your eye.

The million mirrors on each DLP chip roughly correspond to a resolution of 1280x768, and the demo Grant and Yobie showed me was 720p video piped out of a MacBook Air. In one case, it was an off-the-shelf digital copy of Life of Pi. But even with a 720p source and the equivalent of a 720p output in the glasses, I couldn't pinpoint the resolution. The only giveaway was some video compression artifacts from the source footage, which is a factor of bitrate, not resolution. Objects in focus looked much sharper than I would expect from 720p, and with the interpupilary distance adjusted, there was no crosstalk that I could see in the 3D. But more importantly, my eyes weren't strained and I actually could let my pupils comfortably wander and focus on different parts of the image. It was much more like looking at the reflective screen of a movie theater than a big screen TV at home.

Grant told me that image simulates an 80-inch screen sitting 8-feet away from your eyes, which I can't confirm, but I did perceive a sense of distance between my eyes and the image. That's another way of saying that the Virtual Retinal Display doesn't fill your entire field of view, like the Oculus Rift does. The lenses between the DLP chip and your eyes frame the image so that there's a noticeable windowbox around it. I'm told that's because Avegant wants the first generation of these glasses to be practical for legacy media consumption, since it can accept video signals from any HDMI source. 2.35:1 aspect ratio Blu-ray movies don't look good if they're stretched to fill your entire field of view. This is CinemaScope, not IMAX. But wouldn't that be great?

The second demo I was given was of Virtual Retinal Display's head-tracking capabilities for gaming. With the glasses strapped tight to my head using a makeshift GoPro strap, I played a little bit of Call of Duty: Ghosts using head movement instead of the right controller stick. This isn't a game designed for HMD head-tracking, so aiming and directional control wasn't good, but the point of low-latency was made. Like the Oculus Rift, these glasses use a nine-axis motion tracker (gyro, accelerometer, compass) for its head-tracking, but I can't say whether it's as good without direct comparison. One interesting thing is that the 540p render of COD: Ghosts didn't look nearly as sharp as the high-def video footage--it was weird to see aliasing in the game graphics without seeing the pixels of the display.

If the photos of the Virtual Retinal Display make it look hacked together, that's because it's still early in the prototype stage. Avegant has been working on getting the optics figured out, and will unveil their target consumer design next month. The goal, Yobie said, was for glasses that you wouldn't have to strap to your head. They will have to be lightweight--a 10 ounce goal is tentative--and that's with added high-fidelity audio that's not in the current prototype. Another eventual ambitious goal is to make it completely wireless, using onboard power and a wireless receiver to connect it to a video source--even if that's a smartphone. [Correction: Avegant reached out to clarify that wireless was an eventual possibility, but wouldn't be making it in the next prototype or what's coming early next year.]

Another difference between this and the Oculus--you're not supposed to wear it over your glasses. The Virtual Retinal Display has a built-in diopter system that lets you focus each eye independently using a click wheel.

For a product that's only been in development for 11 months, Avegant's Virtual Retinal Display looks well on its way to becoming an interesting consumer product. The company will be at CES in January to show its next prototypes, which could also coincide with the timing of a crowd-funding campaign.

Grant and Yobie didn't leave me skeptical about Avegant's technology--this is genuinely innovative tech that has the potential to solve a big problem with head-mounted displays. But I am still skeptical about whether people want to use HMDs for watching video and gaming, even as just a complement to a big-screen in the living room. When I asked them how they felt about entering a market that has yet to prove itself, Yobie said that it's the kind of challenge he welcomes. Long before his last job as Global Chief Technology Officer for Citigroup, he was an engineer at Lotus working on another unproven technology. It was the spreadsheet.