Digitally Unable to Repeat the Past

An article in the L.A. TImes by Charles Piller begins,

"Carter G. Walker remembers the day her memories vanished.

"After sending an e-mail to her aunt, the Montana freelance writer stepped away from the computer to make a grilled-cheese sandwich. She returned a few minutes later to a black screen. Data recovery experts did what they could, but the hard drive was beyond saving — as were the precious moments Walker had entrusted to it."

The article goes on to lament the loss of digital data even under the government's care, quoting archivists from the National Archives and citing examples such as priceless lost NASA data, and says that W's presidency is likely to leave a less complete legacy than Lincoln's did.

In the mass storage industry, we have known about this problem for a long time, especially the migration problem as different forms of digital media become obsolete. One key problem is more than just hardware is required to read modern media -- complex software is required, too. Even if you could somehow take apart a DVD player a hundred years from now and recreate the hardware, without the firmware, you're out of luck.

The IEEE Technical Committee on Mass Storage Systems (MSSTC), the executive committee of which I'm a member of, sponsors a series of conferences, at which the archival problem is often addressed. Bob Coyne of IBM and Reagan Moore of SDSC are particularly active in this area, especially in being able to find the particular data you're looking for, which starts with tagging it with the right metadata when it's created.

At one conference, we had a presentation from some people interested in creating archival storage in the sense that the Rosetta Stone is archival storage. It was a stainless steel platter with microscopic pits for bits, like a permanent pressing master for a large CD. But it also included a picture visible to the naked eye explaining how to build a reader for the data. Good idea.

"'If we don't solve the problem, our time will not become part of the past,' said Kenneth Thibodaux, who directs electronic records preservation for the National Archives. 'It will largely vanish.'"

Indeed.