Why Converting Pixar Movies to 3D Is No Easy Feat

By Wesley Fenlon

Pixar's head of 3D conversion explains the interesting opportunities offered when bringing back a CG classic.

Our favorite movies, the ones that truly grip us, ignite our imaginations and create characters and worlds that live on in our minds forever. Those worlds expand far beyond the two or three hours of picture and sound captured on film to take on new meaning in our memories or in the public consciousness. They're eternal.

But physically, those worlds are fragile--Martin Scorsese's Film Foundation estimates half of all films made before 1950 have been lost forever. Cellulose film, introduced in the 1950s, slowed that loss, but degradation was still a problem.

By the 1990s, digital technology seemed poised to make film loss a concern of the past. But the new pace of digital advancement presented another wrinkle: Within a few short years, the tools used to make a digital film, like Pixar's groundbreaking CGI animation Toy Story, are obsolete. New software and the jump from 32- to 64-bit hardware could easily render files and assets obsolete, despite the thousands of hours of production time poured into them.

It's a problem Pixar knows only too well, and even movies they've made in the past decade are vulnerable to technological progress. When Pixar's stereoscopic 3D team turns its attention to an older entry in the company's library, they call it digital archeology.

"[We] spelunk back into the old files and assets and make them re-renderable in modern hardware," explains Bob Whitehill, stereoscopic supervisor at Pixar Animation Studios. Now that Pixar's library of successful movies includes a range of modern classics like Toy Story, Monsters Inc., and Finding Nemo, the animation studio has to divide resources between new projects and the maintenance and preservation of existing films. Whitehill's stereoscopic team does both: They work on the stereoscopic 3D for in-production films and the restoration (revival? 3Dification?) of the back catalog.

Whitehill and his stereoscopic team worked on Finding Nemo 3D for about nine months to prepare the film for a theatrical re-release in September. On December 4, Nemo 3D arrives on Blu-ray. Though most viewers will never notice, Finding Nemo 3D isn't quite the same film it was nine years ago (believe it or not, Nemo will celebrate his 10th birthday in May 2013). A scene here has been slightly altered to work better in 3D; a CGI asset there has been tweaked to correct a mistake in the original release.

"We've been trying to talk about it as a recreation of the movie rather than a conversion," says Whitehill. "We don't start with a finished render and convert it. We resurrect the original asset and shot files and essentially refilm the entire movie, with two eyes this time."

But don't worry--despite the potential implications of the word "recreation," only the most eagle-eyed viewers will notice the tweaks made to the original film (other than the added dimension, of course). Star Wars Special Edition this is not.

At nine years old, Nemo's just dated enough to give today's software headaches, and just young enough to clearly demonstrate how complicated Pixar's CGI technology became in the 2000s. Whitehill shared the challenges brought by bringing movies like Toy Story back into theaters for a new audience.

Resurrecting to Re-render

In 2009, Whitehill worked on the 3D re-release of Toy Story, which is practically a dinosaur in the world of digital filmmaking. The fact that it was 14 years old was both a blessing and a curse. "Because it was so old, it was...hardest to resurrect," Whitehill says, "but once we got it up in the software it was easier to render because...of the simplicity of the scenes and CG."

No such luck with Pixar's undersea adventure. "Nemo was hard on both sides," says Whitehill. "It's modern enough to be really complex, but old enough to be difficulty to resurrect."

The recreation could have been much even more difficult. For Brave, Pixar created new animation software called Presto (named after the 2009 short). Brave's director called the studio's old software "a Model T now with new headlights and a drivetrain," elaborating "All these pieces and pipes sticking out....like a Terry Gilliam contraption. Like the craziest Terry Gilliam machine. It worked for everything--layout and rigging and lighting. It was time to build a new animation software system."

Thankfully, Whitehill and his team were able to use Pixar's old software for Finding Nemo 3D. "It took some really bright people who had worked on the original movie to iron out that process," says Whitehill.

Plenty had changed since 2003, but diving back into the film's reefs and shipwrecks was essentially learning to drive that Model T with a new drivetrain. Adapting it to a completely new piece of software would be like learning how to fly a plane. Even so, there were some systems from Finding Nemo that they couldn't completely replicate after nine years of software and hardware updates.

"One thing we tried but couldn't quite perfectly emulate are some of the procedural things put in the film," says Whitehill. He gives an example: the sea grass on Nemo's reefs moved and swayed based on data created by a random number generator. The "seed number," as Whitehill calls it, controlled the movement speed of the sea grass. But there was no way to retrieve the exact digit created by the random number generator for Nemo's final render in 2003. The only solution was to watch the original film and match it as closely as reasonably possible.

In another case, the stereoscopic team had to make the opposite judgment call: Not matching the original film ended up being the right decision.

"In the original movie they took great care in the number of specks and volume of specks...of particulate matter," Whitehill says, referring to the bits of plankton or dirt that floated in Nemo's ocean. "We found in general that when you have that floating stuff in 3D, it feels like there's more of it than in a 2D space, so we could reduce it." The team had to reduce the volume of the specks and rearrange them so that they didn't float in front of the audience's vision and make the entire film look murky.

Most of Whitehill's changes came about in Finding Nemo's framing--how virtual camera angles are set up to depict the action. One scene was tweaked so that a screw would shoot forward out of the corner of the screen instead of leaving frame right; another was adjusted slightly to keep the lanky body of a barracuda in-frame.

"I think we're very fortunate in 3D computer animation because we have so much time with which to set the 3D parameters just so," says Whitehill. "You can imagine with a live action set there are so many variables...[and] with live action lenses, they're incredibly complicated pieces of equipment. Really tough to get left and right eye cameras to match exactly. When you zoom in you might have a different center point and get some vertical disparity or distortion."

By contrast, Pixar can perform as many digital "takes" as it wants reframing a shot in 3D, tweaking the angles and depth until the scene looks just right. Another advantage: With cameras that are computer code and not physical objects, the 3D effect can be absolutely perfect, with no visual incongruity between what the left and right eyes see.

"I get to set the 3D parameters on a shot-by-shot basis," Whitehill says. Directors of photography, eat your heart out.

Techno Troubles, Techno Solutions

For years, directors and cinematographers would shoot a day's worth of scenes and send the film off to be processed. They'd have to wait until those unedited bits of film called dailies came back to see how a shot really looked. Did they get the perfect take? Were the lighting and focus just right? With digital technology, that review process can be done instantly on a camera monitor.

There's no waiting for dailies in CGI animation, but reviewing a completed scene isn't a speedy process, either. In intricate productions like Pixar's, every individual frame of a movie can take hours and hours to go from animation software to fully rendered image, brought to life with detailed shaders and lighting.

Instead of reviewing multiple scenes, animation teams may have only a few frames of new footage to check out every day. On Toy Story 3, the average frame took seven hours to render, and others took up to 39 hours of intense computing from Pixar's server farm.

Here, though, new technology really shines.

"Ballpark, I'd imagine the processor speed is a little over twice as fast as it was when we originally rendered [Finding Nemo]," says Whitehill, "and we have more processors as well....We could look at a shot, render it, view it, and get it the next morning."

Pixar's famous render farm has grown tremendously since the studio made Finding Nemo. It grows larger and faster with every project, and in 2011 Pixar tripled the size of its render farm to produce Cars 2. It's easy to imagine, then, why Toy Story was so easy to re-render once it was up and running in Pixar's modern software environment. Visually, Buzz and Woody's original adventure is dramatically simpler than anything Pixar has made in the past decade.

As each successive film becomes more complex, Pixar faces another challenge: Storing the terabytes upon terabytes of data that its render farm is cranking out. In addition to original animation files, storyboards and art assets, and whatever else Pixar has in its digital archives, switching to stereoscopic 3D doubles the size of each rendered film.

Nemo 3D is actually slightly more than twice the size of its predecessor. Surprisingly, back in 2003 Pixar rendered the movie at a resolution of 1600x900. The new release hits the now-standard 1920x1080. But does that mean Pixar's films are already doomed to be low resolution a decade or two from now, as 4K and eventually 8K TVs replace our 1080p LCDs and plasmas?

Not at all.

"Our software operates independent of final rendering resolution," explains Whitehill, "so our animators and other departments work at various resolutions to suit their needs. Some departments may choose speedy, lower size renders and others choose full size digital film renders.

We do have the ability to re-render any of our films at a higher resolution such as 4K should we choose to in the future."

When that time comes, Pixar's render farm will have to go through another massive growth spurt. And so will its storage space, which Whitehill estimates already exceeds five petabytes, or 5.4 million gigs.

With luck, Pixar's recent films like Brave will be easier to recreate or rerelease a decade from now. "We're trying to anticipate," says Whitehill. "By no means was there a lack of looking forward a decade ago...they did their best in terms of archiving and planning for future iterations of software."

Of course, in another decade, Pixar may be ready to move from Presto to another software platform. A sequel to Finding Nemo, once again directed by Andrew Stanton, is scheduled to be released in 2016. By 2025, someone at Pixar will probably face the challenge of bringing it (and perhaps its progenitor) to life for a new generation of kids. It's going to take a whole lot of pixels.