Assignment #8: Holoportation

Ever since the unveiling of its HoloLens project, Microsoft has come up with a variety of demonstrations of how this portable, augmented reality headset may be useful for businesses as well as consumers. Microsoft Research’s Interactive 3D Technologies group is working on real-time 3D hologram projection of people with the help of HoloLens.

Holoportation makes use of multiple special camera rig setups, placed around a room. These cameras are able to capture the depth information of the objects/persons in the room, thus allowing the system to reconstruct 3D models of these objects/persons. These models are then textured using the real images from the camera, to create high-quality 3D holograms. This information can then be transmitted to any part of the world, allowing the person to virtually teleport to any location in the world, provided the person on the other end is wearing the HoloLens to see the person’s hologram.

Researchers at Microsoft demonstrating the holoportation technology

As can be seen from the demo video clip released by Microsoft, one can not only view real-time holograms of people and interact with them, but one can also record the entire conversation and replay it from any perspective, and even scale it up or down like a memory, very similar to Princess Leia’s iconic distress call to Obi-wan-Kenobi in the movie Star Wars. So is holoportation going to replace video conferencing in the future? Well, not quiet yet!

While holoportation demonstration videos look very lucrative, there are many reasons why this technology is not going to replace video conferencing just yet. Firstly, if you wish to “holoport” yourself, you will need to have special cameras installed in your room. Although the depth information can be captured by having two such cameras installed, Microsoft says that the more the number of cameras, better is the quality of the 3D reconstruction. Even if you have such a setup in your room, someone else with a HoloLens will be able to view you, but you won’t be able to view him, unless you are wearing a HoloLens as well.

Secondly, the field of view available in the HoloLens is very small. This means that the holoported holograms will appear floating in the room instead of standing on the floor due to clipping. This can become very frustrating very fast. Furthermore, there is also the problem of bandwidth. Transmitting all this information will require very high speed internet connection, which can very easily become a bottleneck, if any of the parties have a poor internet connection.

Lastly, there is the issue of tracking the physical space where you are going to holoport. If the location you are teleporting to does not have the required camera setup, or if you are not wearing a HoloLens yourself, you will not be able to know what lies in the location you teleported to. As can be seen in the demo video, there is a chance of your hologram running into persons or objects in the remote location if you have no idea what’s out there. Moreover, there is no haptic feedback when you are in a remote location, which is very essential for effective collaboration between people. Without haptic feedback, you will appear like any other non-tangible 3D object in the virtual space of the remote user.

Microsoft needs to work hard and find innovative ways to address the issues surrounding holoportation. While this technology has a few issues, we should not forget that HoloLens is still under development, and not released for use by enterprises and consumers yet. Have a look at the below video of Alex Kipman, the inventor of HoloLens and Kinect, where he demonstrates holoportation with a NASA employee, and talks about his vision of the project and the future.

Alex Kipman, the head of the HoloLens project at Microsoft, demonstrating holoportation at the TED conference in Vancouver