Last week, Christina Heller (CEO) welcomed me to her cutting edge volumetric capture stage: Metastage (find out more here). Volumetric Capture is a filming technique where tens to hundreds of cameras capture a performance from all angles and then using AI technology, the footage is stitched together to give you a full, 3-D asset of the person. Remember the “Help me” hologram of Princess Leia sent to Obi Wan Kenobi (watch it here)? Well, that is the type of asset that is created with volumetric capture (or volcap for short). You get a full rendering of a person and when that asset is used in AR or VR, you can walk around and interact with a full person, not just a flat screen. The possibilities are exciting!

Another application of volcap is 2D special effects (SFX). If you remember the scene from The Matrix where Neo dodges the bullet, that was volcap technology in its early days used as SFX and they called it “Bullet Time”. Today, Christina Heller continues to pioneer the evolving technology with her Metastage studio.

I spoke with Christina about volcap and I was most impressed with her vision for the future. Christina comes from a media production background, but in 2014, she spontaneously decided to go to Sundance. While she was there, she experienced their New Frontier program, which showcases pioneering work in the VR space. Inspired by the showcase, she realized that she could make her mark on the world in this technology space. And so, she formed Metastage. 

Her vision is to have volcap Metastages all around the globe so that many more people have the ability to use and innovate storytelling with this technology. They are currently expanding and setting up a new stage. More than that, Christina has a contagious passion for new use cases for this technology.

I spoke with Christina about reality in the Metaverse (a term coined by Neil Stephenson in Snow Crash that thanks to Facebook’s Meta, that term “Metaverse” has entered the public lexicon). There is a distinct lack of filmed reality in virtual reality. Many projects are animated due to the flexibility of animation, but sometimes we get into uncanny valley territory, where the animation looks close to reality in a creepy way. Christina says that the term uncanny valley has helped Metastage to distinguish themselves. They record human beings and so we have all the nuances, facial tics, movements and subtly that we are used to when watching humans. Thus, Volcap does not fall prey to the uncanny valley effect. 

The realness of capturing humans does come with limitations though. If a project calls for a branching narrative, where the viewer has a choice to select one option or another, the trick to filming in volcap is to find a neutral position to start and end each performance on, so that there can be a seamless flow in the performance regardless of what choice the viewer makes.

All these technical details really geek me out. But the most important thing is that people find applications for this technology. Christina acknowledges that she is searching for more ways to incorporate volcap into the traditional 2D movie and TV SFX workflow. You can film an actor from all angles with volcap and then design wildly difficult shots in the post production process. It can be much more affordable to film with volcap to get tricky shots, as opposed to all the dollies or drones it would take to capture the shot in physical reality. So, volcap can be used widely in 2D projects. 

Volcap can be used to preserve precious memories and experiences. Think about your Grandma and all her life experiences. What if you could view memories you share with her? With interviews in volcap, you can film people important to you and capture memories in a better way.

For us forward thinkers, volcap can be used in ways that push the boundaries of storytelling. This is my passion and I will be one of the people to innovate storytelling specifically for VR. Environments in VR are very important, and I believe that with environments like a stage, we can use volumetric capture to get theater type performances only in VR, where you can interact with the characters in new ways. We can combine branching narratives, framed entertainment (I use 3D movie techniques) and volcap to change storytelling and make use of all the strengths of VR: that feeling of full immersion. The future is coming fast, and I’m excited about the change that art and technology will bring. 

Learn more about Metastage here: https://metastage.com/