The Promise of Memex Revisited

Editor's Note: This week, we're looking at the future of storage with virtualization, enterprise flash storage, and the implications of the cloud. Since we're looking forward, it also makes sense to look back with four awesome videos about the history of storage and also the way computer scientists envisioned scalable, applied storage at the dawn of the computing age.

It's likely you've heard of the Manhattan Project, but you've probably never heard of Vannevar Bush.

Vannevar Bush was a computer scientist, the first scientific advisor to the President of the United States, and the director of the wartime Office of Science and Research Development. To put things into perspective, the Manhattan Project was just one of the projects that fell under him (albeit the largest in his portfolio). He managed teams attempting to crack Axis codes, labs developing radar, and also investigated the problem of storing, cataloging, and retrieving the enormous amounts of data created by the government.

In the 1940s, bureaucrats found themselves under a mountain of paper. They decried the problem as information overload. Dozens of new government agencies sprung into existence, requiring interagency coordination and more paperwork. The federal government entered the war with $650,000 worth of printing equipment and owned $50 million worth within a year.

As a leading scientist in analog computer design, Bush saw a solution to the issue of information and data storage in automated cross-referencing and retrieval using a device called the Memex.

The Memex

The Memex was a theoretical machine that Bush first outlined in 1939 with an article called "Mechanization and Records." The way the machine functioned can be found in the name itself. Some maintain the name combines the words "memory" and "index," while others think the device was named as a "memory extension."

It wasn't until after the war that he published his famous article in the The Atlantic titled, "As We May Think," which described the machine in detail.

Bush envisioned a desk with a screen and buttons that would search a library of information stored within the machine itself. The device certainly wasn't a personal computer or even a mainframe, despite its appearance. He didn't mention a database by name, though he eluded to storing records. As described, it also wasn't networked to any other similar devices. Bush's Memex was prophetic of a single, now-common feature the hyperlink.

"With one item in its grasp," Bush writes, "it snaps instantly to the next that is suggested by the association of thoughts, in accordance with some intricate web of trail carried by the cells of the brain." In today's terms, he was almost talking about a hyperlink.

The "almost" is important here. Bush's Memex was suggestive of hyperlinks because he proposed an associative model for retrieving information. Indexing functioned alphabetically or hierarchically through parent-child classes and subclasses. Rather than cataloging the information hierarchically, the Memex user could associate (in our terms) keywords with the piece of information, and then the machine would organize such information according to those associative trails.

Despite the fact that he makes no mention of the missing pieces such as the metatags, search, or databases, Bush's theoretical information console advanced an idea that machines could organize and connect pieces of information in a meaningful way an idea that computer scientists built upon for the next 30 years.

"As We May Think" set the doctrine for the new age of computing.

The Mother of All Demos

Decades later, in December 1968, American engineer Douglas Engelbart famously addressed a meeting of computer developers in San Francisco with a statement easily lifted from the annals of Philip K. Dick.

"If in your office you as an intellectual worker were supplied with a computer display back up by a computer that was alive for you all day and was instantly responsive to every action have, how much value could you derive from that?"

He demonstrated the use of a mouse on a massive screen video-conferenced from Menlo Park, 30 miles south from the auditorium, to operate a system featuring word processing, copying and pasting, and simple searching. Engelbart brought reality to the conception of a hyperlinked network of information in a user-friendly way.

As remarkable as it might seem to see Engelbart to operate a modern-like system, Engelbart's legendary demonstration of a graceful system in action was merely suggestive. It would take more advancements in hardware, software, and culture to create the environment we know today where we can snap instantly to the next hyperlink that is suggested by the association of thoughts.

Thanks to advancements in storage, and the dedicated individuals who maintain these systems flawlessly, the hope of the Memex and Englebart's demo are reality today.