Quantcast

16-bit Time Capsule: SNES Emulator Makes a Case for Software Preservation

By Wesley Fenlon

bsnes creator byuu leads by example, devoting seven years and over $10,000 to immortalize an important piece of gaming culture.

In August of 1991, Nintendo unleashed the Super Nintendo upon the United States. The 16-bit console gaming war had begun, and the SNES charged ahead with a 3.58 MHz central processing unit. That wasn't exactly high technology at the time--the top-end computer processor on the market was Intel's i486DX, which ran at a then-amazing 50 MHz and cost upwards of $3000--but the Super Nintendo's $200 launch was a bargain by comparison. Twenty years later, processors have advanced from the 486's record-setting million transistors to Ivy Bridge's 1.4 billion...and yet, to accurately emulate the humble 3.58 MHz Super Nintendo on a PC, we need computers running about one thousand times faster than the little grey console's humble CPU.

Perfection, it turns out, comes at a high price. But the payoff, at least in one man's eyes, is the preservation of an important piece of video game history. The Chrono Triggers and Super Mario Worlds of the Super Nintendo era will live on for decades, but what about the inner workings of the custom 16-bit Ricoh processor designed just for Nintendo, or the early polygonal computations of the Super FX? Doesn't that engineering merit preservation so that 50 or 100 years from now we can look back on 1990s computer hardware and know exactly how it functioned, and importantly, how the games played?

"Although it's already becoming difficult to obtain the old tech needed to reverse engineer SNES behavior (parallel ports, floppy drives, copiers, etc), I believe working SNES hardware will endure for probably 50-100 more years," wrote byuu, the developer behind Super Nintendo emulator bsnes, in an email interview about the project he's been working on for the past seven and a half years. "What worries me ... for the SNES is if anyone will care 20-30 years from now."

byuu began developing bsnes in 2004 with a seemingly impossible goal: "perfect" emulation of the SNES, with no cut corners, no speed hacks, and true low-level code reproduction of the original hardware. In March 2012, bsnes hit an unprecedented milestone: byuu and a small but dedicated community were able to reverse-engineer every single SNES coprocessor (like the Super FX chip used for Star Fox's 3D graphics). In a series of giant leaps towards the technological preservation of the Super Nintendo, this was perhaps the most significant.

With its unusual focus on uncompromising accuracy over performance, bsnes is a little-known champion of software preservation, a growing issue for the video game and computer industries. Our old software, trapped on attic-bound discs and CDs, is rotting away. With every new era of computing technology, we abandon an outdated storage medium and make it just a little harder to access aging data. Before long, hardware and software not deliberately preserved will be lost forever. bsnes demonstrates what a massive commitment of time and money preserving half a decade of culture can be. Here's how byuu did it.

Who needs another SNES Emulator?

byuu, who shares an Internet nickname with the protagonist of Squaresoft's Super Nintendo RPG Bahamut Lagoon, began work on bsnes years after Snes9X and ZSNES had given millions of gamers a way to play pirated ROMs on low-powered Pentiums. Neither emulator focused on closely reproducing the Super Nintendo's code. Instead, they used game-specific hacks and tweaks to lower emulator system requirements. byuu took the opposite approach and focused on writing the most accurate interpretation of the original SNES hardware he possibly could.

Without proper emulation of mid-scanline raster effects, the plane's shadow in Air Strike Patrol disappears. Image credit: Ars Technica.

By 2010, bsnes was easily the most accurate SNES emulator on the Internet. Defending his project took its toll over the course of six years: byuu found little interest in a CPU-intensive emulator outside the small fanbase that had actively contributed to bsnes' development. Fan translations and ROM hacks that only worked in ZSNES were the final straw. He vented his frustration in a 2010 article and chose to abandon proselytizing his emulator across the web (until last fall, when byuu wrote an article about accurate emulation for Ars Technica).

Though bsnes was "pretty much finished" by 2010, byuu's next two years of work would set a high bar for software preservationists to live up to.

Coprocessor Quest

Gamers who grew up with the Super Nintendo know all the weaponized buzzwords of the early 90s. The GameCube and Wii have seen Nintendo push mass-market appeal over technology, but in the 16-bit era they fought back against Sega's "Genesis does what Nintendon't" ad campaign by hyping up the Super FX chip and Mode 7 graphics. Sega's "blast processing" slogan held a kernel of truth: the Genesis CPU was clocked at 7.67 MHz, twice as fast as the beating heart of the Super Nintendo.

The SNES needed help pulling off more complicated graphics, so Nintendo planned ahead. They built in support for coprocessors or "enhancement chips," which were embedded in game cartridges to perform additional processing the CPU couldn't handle. As always, Nintendo found a way to keep its base hardware as inexpensive as possible.

"The SA-1 and SuperFX [added] roughly twice the processing power," says byuu, describing the Super Accelerator 1 used in Super Mario RPG and the well-known chip behind the graphics in Star Fox and Yoshi's Island. An article on byuu.org lists every coprocessor used in a Super Nintendo cartridge. He explained the differences to me in more detail. For example: "The DSP-1 provided many 3D coordinate calculations, useful for Super Mario Kart and Pilotwings," while "The Cx4 provided geometry calculations and wireframe rendering for Mega Man X2 and X3."

Photo Credit: Flickr user miskan via Creative Commons

Where the Super FX and SA-1 were programmable sidekicks for the SNES CPU, a range of coprocessors called DPSs (digital signal processors) were pre-programmed to run unique code for individual games. Until byuu turned his attention towards the the coprocessors, they had only been emulated through high-level emulation, which simulates functionality rather than reverse-engineering and running old code on a new platform.

"I knew from the beginning that the HLE method of reverse engineering the functions of these coprocessors would never lead to perfect emulation," he explained. "Even with over a dozen years of research from another dozen genius-level math experts, only the DSP-1 and DSP-2 were considered to be emulated bit-perfectly (meaning: feed in X, get exactly Y value out.) But even then there was no attempt at all at simulating time on these chips. That is to say, imagine you asked to perform a very complex calculation. With HLE, the SNES CPU will see the coprocessors completing any task instantaneously. Whereas LLE will have to tell the CPU it is busy while it performs the computation."

An electron microscope scan of DSP-1B. Photo credit: Byuu.org.

LLE is the key to accurate emulation: by running original code, emulators can reproduce the exact behavior of real hardware. The Super FX and SA-1 were (relatively) easy to reverse-engineer, but the DSPs were proprietary chips, making their architecture and code all but impenetrable. bsnes found a champion for its cause in 2010 when an electrical engineer from the MAME emulation scene offered to decap the DSPs, melting away the epoxy covering with nitric acid, and scanning the chips with an electron microscope. The "savant," as byuu describes him, went by the handle Dr. Decapitator.

"It wasn't until 2010 that anyone even dreamed of having the possibility of extracting the program ROMs from these cartridges," byuu wrote. "This was only made possible thanks to Dr. Decapitator appearing on the scene...There is a sense of wonderment and remorse I feel about it all. It took me three weeks worth of effort (with the help of cydrak and segher) to perfectly emulate all eight coprocessors. This has rendered the aforementioned dozen years of research from a dozen individuals, which only rendered five coprocessors playable in the end, obsolete. I can't imagine how disheartening that must be to all of their efforts."

By March 2012, the bsnes community had raised $2500 to pay Dr. Decapitator to decap and scan the chips. From there, a small force of coders, including byuu, figured out how to convince each DSP to give up its code. "There wasn't anything particularly magical about my involvement," byuu wrote. "It was just a matter of attracting the right individuals who could help obtain the carts, donate to pay for decapping work, decap and dump the DSP programs, reverse engineer the instruction sets, and fix bugs in the resulting LLE. I was very fortunate to be able to find so many great people to help me."

With every coprocessor emulated, bsnes was finally able to claim complete support for the entire library of officially released Super Nintendo games. In some cases, switching from HLE to LLE emulation only fixed small errors. "The imperfections in the algorithms manifested in slight oddities, such as improper sprite ordering in [Mega Man] X2 and X3," byuu wrote. In others, they made games never before playable in an emulator fully functional in bsnes. A Japanese Shogi game was the final piece of the puzzle.

The universal asynchronous receiver/transmitter byuu used to dump ROM data from the ST018 DSP, installed in a Japanese Shogi game, to his PC. Photo credit: Byuu.

Nearly three years of effort went into reverse engineering the SNES DSPs and adding their code to bsnes. Let's put that into perspective. After about 15 years of active development, the SNES emulation scene finally has a low-level emulator that's nearly representative of the original hardware. The Super Nintendo is over 20 years old, and every successive console generation makes Nintendo's 16-bit programming look like child's play. How can we hope to preserve that technology for the sake of history if 1) it's so dizzyingly complex and 2) only extremely dedicated fans like byuu are even bothering to try?

Something has to change.

Emulation, Copyright Law and the SNES Preservation Project

Despite the fact that the Super Nintendo DSPs are between 15 and 20 years old, most of them remain closely protected by their creators. "I cannot distribute the DSP program ROMs, as they remain copyrighted," byuu responded when I questioned him about the legal issues emulation has long skirted. "I couldn't even get Renesas (formerly NEC) or Hitachi to provide me with their technical documents on the processors they once sold to the public ... despite the fact that there hasn't been any money to be made on these old programs for at least a decade now."

Every SNES game released in the United States. Photo credit: Byuu.org

Alongside bsnes' development, byuu has been collecting every Super Nintendo game released in the US for what he calls the SNES Preservation Project. The ambitious goal: to scan the boxes and manuals of each game and dump the data of every cartridge, creating the best possible digital archive of the American Super Nintendo library. Sound expensive? It is: even borrowing some games, he's spent over $10,000 amassing the North American library of more than 700 games.

byuu follows the law and believes strongly in the ethics of emulation. Emulators should be open source. Neglected products shouldn't be entitled to lengthy copyrights. He can't upload any of the scans or cartridge dump to the web, though he does plan to publish SHA256 cartridge hashes, which will allow others to verify their cartridge dumps, and XML memory maps, which document exactly how every SNES chip was laid out. byuu's ethics statement references a study by Rufus Pollock, an economics professor at the University of Cambridge, which estimates the "optimal" copyright term at around 15 years.

In the United States, copyright lasts 70 years after the author's death. That's a bit of a gap.

Why does the 10 MHz processor in a Super Nintendo cartridge need to be protected by copyright for a century when it's only financially valuable for half a decade? Why does copyright law on a piece of software extend past our ability to run and preserve that software with compatible hardware?

Every year the computer industry ages, early software edges closer to extinction. The divide between copyright law and copyright ethicality--the divide between 19th century law and 21st century common sense--widens.

Photo credit: Jordan Mechner

Benj Edwards' article "Why History Needs Software Piracy" outlines how thousands of programs have been saved from extinction thanks to illegal distribution through file sharing and abandonware websites. Edwards estimates that about 23,000 games have been released on floppy disks or cassettes since the 1970s. Floppies have a shelf life of about 30 years, meaning their time on this Earth is nearly up.

Just last month, gaming icon Jordan Mechner discovered the long-lost disks holding the Apple II source code for Prince of Persia. The floppies hadn't been touched in 22 years; when Mechner announced on Twitter that he'd found the source code, an Internet archive employee volunteered to fly from New York to LA to help retrieve the data on the disks. The process required almost as much specialized hardware as decapping a SNES coprocessor, but it was successful. You can download the source code for free on Github.

"From a preservationist point of view, the POP source code slipped through a window that is rapidly closing. Anyone who turns up a 1980s disk archive 20 or 30 years from now may be out of luck. Even if it’s something valuable that the world really cares about and is willing to invest time and money into extracting, it will probably be too late." - Jordan Mechner

In another 20 years, all of our floppy disks and magnetic tapes may have decayed beyond digital legibility, data literally turning to flecks of once-magnetized dust. A decade after that, storage mediums we rely on this very day may have reached the end of their worryingly short lifespans.

"FlashROM used in Nintendo DS cartridges are only rated for 10-20 years of life...Far less than MaskROM used by the SNES era," byuu wrote when I asked about the eventual disappearance of SNES hardware. "More complex systems lead to quicker faults as well. You don't hear much about red rings of death on the SNES, for example. And nowadays there's all the DRM, the digital-only downloads and post-sale game patches, and the upcoming restrictions against resale, and I see a very bleak future for preservation."

Even ignoring the inevitable nightmare of archiving games that have been patched and expanded with downloadable content--and setting aside the headache of online game servers being shut down after a year or two of operation--there are vast libraries of games on CDs, DVDs, cartridges and FlashROM cards that are slowly but surely shuffling towards a bit rot graveyard.

Most, if not all, of those games have been pirated and put online. As bsnes has shown, preserving functional hardware code for future computer systems is the greater challenge. And who's willing to put in a decade of effort for something that won't make them a dime?

"Although I cannot predict an alternate future's timeline, I do truly believe that had I not taken the charge to get these SNES coprocessors decapped and emulated, that it never would have happened," wrote byuu. "As far as the DS goes, I think that a lot of people really want to emulate their favorite childhood systems for nostalgic reasons. ... I suspect that we will see more interest in DS emulation as the youth that grew up with the system reach maturity and obtain CS degrees and such."

Other emulation projects, such as the Wii and GameCube emulator Dolphin, are vastly complex pieces of software that make games playable and even improve them through high resolution rendering. "There's just no way we'll ever have a bit-perfect, low-level simulation of the Wii," byuu wrote, but he admitted "at the end of the day, the most important thing is simply being able to play the game at full framerate."

Wii emulator Dolphin can render games at resolutions of 1080p and above.

Seven and a half years, 89 releases and 10 decapped DSPs after bsnes was born, byuu still sees it as an unfinished project. "We've achieved 100% bug-free compatibility with official games," he wrote. "At this stage, it's more about preserving what the hardware was capable (and not capable) of."

Even if byuu never touched bsnes again, the Super Nintendo is more safely preserved than the Genesis, or the Turbografx-16, or countless other game consoles. Historians will be able to look back on the engineering performed at Nintendo R&D 2 in the late 1980s through the window of bsnes--or some future software built from byuu's source code--and see almost exactly how this one video game console functioned. And 100 years from now, at least some of the ROMs uploaded to the Internet will actually be legal.

"This [SNES Preservation Project] should have been done in the late '90s, but I was a kid then without the necessary income," wrote byuu. "So I do what I can now that I have a job and the requisite income to do so." It's time for the gaming industry at large to do the same. With floppy disks nearing extinction and the FlashROM of some Nintendo DS carts already closing in on middle age, there's only so much the diehard emulation community and software pirates can preserve before things start disappearing for good.

"If you see strict DRM and copy protection that threatens the preservation of history, fight it: copy the work, keep it safe, and eventually share it so it never disappears.
Some people may think ill of your archival efforts now, but they’re on the wrong side of history: no one living 500 years from now will judge your infringing deeds harshly when they can load up an ancient program and see it for themselves." - Benj Edwards