Oculus Connect 4 Chat with Nate Mitchell, Oculus' Head of Rift

By Norman Chan

Top takeaways from our annual check-in with Nate from Oculus at OC 4.

Oculus Connect 4, Facebook's developer conference for VR, is wrapped, and we're still digesting all the information we learned and demo time with the Santa Cruz prototype we experienced during our visit. While Oculus didn't have anything to say about a follow-up to its flagship Rift headset, the announcements it made about standalone VR hardware left us with plenty of questions about the company's product strategy, state of technolgy R&D, and approach to the spectrum of virtual reality experiences on the path to getting a billion people into VR. We were able to ask some of those questions in our annual check-in with Oculus' Head of Rift Nate Mitchell--a lively and enlightening conversation that's a highlight of our Oculus Connect coverage. But policy changes this year prevented us from filming the interview, so we shared some takeaways in our event recap video. In reviewing our interview notes, several points stood out that we didn't convey or stress enough in our day-of recap, so we're sharing those here for clarification and posterity.

We started off asking about Oculus Go and the Santa Cruz prototype and how they fit in a future product lineup for Oculus. Nate talked a bit about how the feature set for Go was determined, and how it, along with Santa Cruz, are on a mobile product track that's separate from Rift. Said Nate, "As you can see with Project Santa Cruz, that may not be our only offering in the standalone product category. You can imagine over time, a good, better, or even a good, better, best approach. But sort of like the most affordable, best experience standalone device you can build, we think Go really competes right there."

Oculus was also coy about the hardware running Go, and its performance targets. For example, would Go offer the equivalent performance of a Samsung Galaxy S7, S8, or better? Nate would only say that it would be comparable: "You can just think of it roughly as equivalent to Gear VR. Roughly [Gear VR as it is shipping today]. I mean, we support a number of past generations of Gear VR today, but you can imagine it's on the same compute envelope as Gear VR."

"Not all of the hardware's locked down" was the reason given for not sharing information like the screen refresh rate on Go, though we suspect it'll be 60Hz like on Gear VR. Also unclear was whether the Android UI would be surfaced at all to users, or the interface when putting on the headset. "We are committed to making it an even better experience than Gear VR. Which is not a very high bar, but a really good experience. So we're doing a lot of neat stuff that we're excited about," said Nate.

On the topic of the Santa Cruz prototype, even though Nate explained that it was on the same standalone VR product pipeline as Go, we asked about how Oculus was thinking about controller parity with Rift and its Touch controllers, especially since porting Rift games to Santa Cruz is something Oculus is already doing themselves (with Dead & Buried). Nate acknowledged that there are differences and feature overlap, and that the goal is hand presense. "We want them to disappear in your hands. Same input team for the most part. It's almost identical as working on the prototype Santa Cruz controllers. In general, we're not going for one-to-one [parity]. At least, as far as we're showing today, one-to-one compatibility. But, of course, input fragmentation is something we take super seriously and we're thinking about all the time. I was saying to someone else, I think we spend as much time designing our input devices as we do the actual head sets themselves. So it's something we're thinking a ton about and we're figuring out how we can make developers most successful, trying to have as much standardized input as possible across the platforms."

Something we pushed him on was the use of touchpads versus thumbsticks in the Santa Cruz prototype, even though the two Santa Cruz demos we were given at Connect didn't make use of the touchpad. Nate clarified that the hardware isn't final for the controllers: "I was a big proponent of thumb sticks and buttons, there's a lot of exciting stuff you can do with touch pad. We're experimenting with that. The main point, which I think you had me articulate very clearly is that we haven't made a final decision on what a Project Santa Cruz input might look like. But there is a lot of awesome internal debate as we multiple teams prototyping a lot of different input sets on what the future should be. We're building all sorts of experiences internally to see what really feels good. I don't think Touch cracked the code. We want to keep pushing forward."

Santa Cruz's most notable feature isn't its controllers, but its inside-out positional tracking. And when it comes to inside-out tracking, we had some concerns about the tracking range offered by its four cameras, and were curious whether Santa Cruz had the potential to allow for multi-roomscale experiences offered by simultaneous localization and mapping (SLAM) technologies. Nate emphsaized that the goal was to be able to toss a headset like Santa Cruz in your bag and pull it out somewhere else to enjoy 6DOF VR without any setup. But he also acknowledged that it's not perfect: "The four camera system that we have on Project Santa Cruz, we feel like is almost like a minimum bar in terms of the overall tracking volume that you want. It's not perfect. We're still doing a lot of work to tune the tracking system. You may be able to break it in there, but also there's more tricks we can do in software using IK and other motions to basically predict what you're doing."

While reiterating that Santa Cruz is only a prototype today, and a consumer product that might come far out in the future will only improve, he also made the point that full 360 coverage for tracking isn't essential. "The number of times you [need full 360] is very, very, very small. We have a good sense of that, actually, what people are doing broadly. The space that they use on Rift. But there're certain contortions you can't get your body to make and I think what you'll see in Project Santa Cruz the demo is all the motions that you want to make, for the most part, you're good. It's not perfect, but it's only going to get better."

And finally, we of course needed to ask Nate about Rift, and how, if standalone mobile VR will have a positionally tracked option, Oculus thinks about the differences between mobile and PC-based headsets. For example, is the difference between those product lines tethered vs. untethered, or computing power? Nate said that it was all about compute power. "I mean, the differential between these two platforms is going to be substantial. These mobile compute devices versus PC compute devices. Carmack said in his talk today that PC horse power today is probably not coming to mobile. Moore's Law will run out and we will be on this two track path. Similarly, when it comes to [mobile], power is a big part of the constraint there. PC is going to continue to move forward. PC's are going to continue to push on. We're going to have flexibility in compute. We're going to have more flexibility in overall budget of these systems, we're going to have flexibility in mounting of trackers and sensors. We're going to have flexibility just in terms of developers bringing experiences to the ecosystem as quickly as possible, which PC is famous for."

And if standalone/mobile VR is going to achieve some feature parity with Rift, we wanted to know if better graphics was going to be enough for Rift 2 to be compelling, or if it needed to offer completely new experiences. "I don't think it has to be... it depends on how we frame up Rift 2. Rift 2 can be super compelling to people without being a totally new experience. There's space in the market today for a next generation VR headset with just higher resolution, wider FOV, potentially wireless, that appeals to these enthusiasts. No one's brought that product to market yet, maybe for a variety of reasons. I also think to your point, there's also a lot of appetite for a completely transformational experience that changes the game with new technologies. Bringing more of you in the experience or some of the stuff we've talked about with mixed reality. We're pushing forward on a lot of different vectors. We aren't talking about new Rift hardware today, but it's something myself, Brendan, and the PC team is working on all the time."

In terms of timing, we also asked Nate that if they weren't talking about Rift 2 at Oculus Connect 4, if that meant that it's further out than a Santa Cruz-based product. Unsurprisingly, we couldn't get a definitive answer on that topic: "That's a really good question that no one's asked me, but I don't think we have anything to share on that front."