How to Build a Holodeck

Admit it, you want a holodeck. I know I do. What could be better than the Star Trek ideal of interactive entertainment? Imagine being able to explore an immersive, limitless world.

But before I get ahead of myself, we need to define what a “holodeck” actually is. First seen in the pilot episode of Star Trek: The Next Generation, a holodeck is a large room with one entrance/exit. Its walls are covered in electronic “emitters” that project light into patterns – or holograms – and these patterns of light are made solid through the use of force fields that conform to their position and size. When people are inside, they are constantly walking on a force field that tracks their movements, making it appear that they are walking vast distances when they are actually walking in the same place. From this comes the definition: a holodeck is a room that creates objects, people and images on the fly and has no restriction on time and space. If you’ve never seen Star Trek (and how could you not?), this video provides a good example of the technology in action.

It’s clear why this would be the ultimate destination for gaming. No longer would there be buttons or controllers or anything of the kind, just a room with infinite possibilities.

How would one go about building such a room? While we have access to holograms, they are often simplistic, without definition and realism. In addition, even if we could easily produce true-to-life holograms, we can’t create force fields capable of forming light into a solid shape. No, we’re going to have to get creative. We need to recreate the experience while staying grounded in reality. To do this, we’ll have to cherry pick from currently existing and upcoming tech.

Let’s break down what a holodeck is, then try to build it in order:

1. Limitless movement in any direction
2. Photo-realistic scenes and characters
3. Complete auditory and olfactory immersion
4. Automatic creation of objects and locations based on programmed simulations
5. Interactions with all objects, including the ability to pick them up and move them

The closest we have to a totally immersive environment right now is this contraption, which I’ll call the hamster ball. While it does cover movement, you’re still limited by the devices attached to you. There are the VR goggles, to start, as well as a full bodysuit. The ball itself won’t even give the feeling of scaling a hill or moving on rough terrain or in water, since it will only work as a semi-flat surface to move across. The advantage this system affords us, however, is the movement tracking system, which connects the virtual world to the physical. We can nab that, to start.

Now there is tracking, but no free form movement. For that, we turn to Virtual Space Devices, Inc., and their Omni-Direction Treadmill. While not a perfect solution, this is the precursor to complete movement, including uphill. While something like stairs would probably be out of the question, the manufacturer has said that tilt (like for an uphill climb) could be accomplished using “linear actuators attached to the frame.” Combining the ODT with the movement tracker, as well as future development on both, and we have “limitless movement in any direction.”

What we need now is the ability to walk around somewhere beautiful and photorealistic. While computer modeling has progressed a great deal in the past 20 years, fooling the human eye is still a difficult task, especially when it comes to creating people out of thin air. A much better solution is to not try and outdo the real world, but to use it. Many full-body scanning systems have made photorealistic characters for major entertainment, like the Matrix sequels, Batman Begins, King Kong and, yes, even game properties. By scanning the objects and people from the real world and capturing important parts, like faces, from every direction light can pass, it captures a “perfect-fidelity” image of the object. Consider this image, which was created by scanning actress Jessica Vallot using USC Institute for Creative Technologies’ “Light Stage 2” scanner. That’s not a photograph, but a digital reproduction of the face, as shown in their demonstration video. With some tweaking, we’ll have “photo-realistic scenes and characters” to populate our holodeck world.

Third on our list is probably the easiest, though not without some challenges. If the world looks right, it also has to sound and smell right. Sound is arguably not an issue, since surround sound systems have reached peak efficiency and could be implanted into the environment, as well as placed within the characters you speak to (more on that later). Smell is trickier, since there’s nothing right now that automatically creates odors, but there are minds at work on it. The Digital Scent blog catalogs these developments, the most promising of which is an old technology from AromaJet.com. Essentially, mixing and matching individual fragrances into combinations could create hundreds, possibly thousands, of aromas. Imagine going through the mines with Gordon Freeman, hearing the critters all around you and smelling the rock and coal filling the place.

Of course, this would all be worthless without the ability to interact with this world. Thus we arrive at the crown jewel of the holodeck: Claytronics. The promising technology, while still technically in its infancy, has been under development at Carnegie Mellon University for a while.

Claytronics shapes molecule-sized robots (called “catoms,” a portmanteau of “claytronic” and “atom”) with static fields, creating a “dynamic, 3-dimensional display of electronic information.” Several papers have already been published on the concept, my favorite being “Adhesion and Anisotropic Friction Enhancements of Angled Heterogeneous Micro-Fiber Arrays with Spherical and Spatula Tips.” I don’t know what the hell it means, but it sounds impressive.

When completed, the catoms will be able to form tangible objects. They may even reach a stage where the objects will feel like whatever object they’re mimicking, perhaps even water. It’s no wonder Carnegie Mellon dubbed it the “Synthetic Reality Project.”

The maturation of Claytronics is years, perhaps decades away, unfortunately. Thus far, the most promising step has been basic movement of small (but far from molecule-sized) catoms, sliding along one another’s surface. Still, even this display is an exciting one, even if it sounds dirty.

Now we have all the pieces:

1. An Omni-Directional Treadmill for free movement
2. Photo-realistic scenes and characters based on advanced scanning techniques
3. Complete speaker system integration and scent dispensing combinators that allow the world to sound and smell real
4. Automatic creation of objects using Claytronics …
5. … and freedom to interact with and move them at will.

Of course, we must also build walls designed to display realistic backdrops. The University of Calgary designed and produced a kind of projection system for walls called CAVE. While currently used for biometric images of the human body, it could be retrofitted to display a real-time world around the participant.

Engineers would have to work to fit all these items together inside a space the size of a medium-sized garage, while teams of computer programmers would create the interaction between the catom programs, the world-projection systems and the movement trackers. Then, after years of testing and tweaking, we would have built the first holodeck.

Much like the first generation iPods, it would be clunky and not work exactly as it should, but the income from such a device (probably in the hundreds of millions) would quickly increase research and development, eventually leading to the near-perfection that Star Trek showed us.

This was just a thought experiment, but there have to be engineers and designers working toward this goal even now. And, if not, take this idea and run with it! I want to experience a holodeck before I die. You’ve got some time with me, barring a horrifying encounter with a cross-town bus, but you shouldn’t wait. Think of all the wonders we could create.

I’ll be waiting.

Tom Rhodes is a writer and filmmaker currently living in Ohio. He can be reached through Tom [dot] Rhod [at] gmail [dot] com