Virtual Production Is Off And Running, So You Better Keep Up

Are you familiar with the term “virtual production”? If not, fear not. This blog article is intent on deciphering today’s hottest production term for you, considering how virtual production will continue to shape the mediamaking practices of cinema, television, and more. And, while visual technology company IKIN does not currently have a direct stake in virtual production, we’ll consider how its ties to cornerstone game engine company Unity and their recently acquired VFX subsidiary Weta Digital open doors to the future prospect of integrating IKIN tech with virtual production.

First Off: What Is Virtual Production?

The Virtual Production Glossary is an evolving database supported by industry staples, including the Visual Effects Society, the American Society of Cinematographers, Epic Games, and Netflix. It was built as an educational resource to standardize the growing lexicon of virtual production terminology. According to the Virtual Production Glossary, virtual production “uses technology to join the digital world with the physical world in real-time. It enables filmmakers to interact with the digital process in the same ways they interact with live-action production.” Thus, it involves an innovative combination of tools and production practices that allow media makers to collapse the digitally-rendered processes of post-production with the tangible processes of principal photography. As such, it also encompasses several different industries, which is one of the reasons why virtual production and its associated terminology can remain–at times–ambiguous.

Some Specific Examples Of Virtual Production

To further understand virtual production, we can consider some of its more recognizable practices, like performance capture, world capture, and simulcam.

Performance Capture

In performance capture and the similar motion-capture (or mocap), specialized bodysuits record the live moments of actors to combine them with digitally rendered characters. This technology was made famous through the convincingly real mocap characters Golem from the Lord of the Rings films, Kong in Peter Jackson’s King Kong, and the primate leader Caesar from the most recent Planet of the Apes movies–incidentally, all of which were played by pioneering performance-capture actor Andy Serkis.

World Capture

World capture (sometimes referred to as “reality capture” or “scene digitization”) allows for easy digital mapping of real-world spaces, like shooting locations and sets. This process relies on a combination of LIDAR scanning, photography, video, and other references to realize these physical spaces in digital form.

Simulcam

Simulcam allows media makers to visualize digital characters and digital environments in real-time. In essence, this is an AR technique that maps digital content convincingly into physical environments.

Virtual Production Is The Talk Of Tinsel Town

Individually these are impressive technologies, but they really shine when combined together to create a beautiful marriage of the physical and digital elements of mediamaking. Weta Digital, one of the leading pioneers of virtual production, points to their work with director James Cameron on Avatar as a pivotal moment for this industry. On that film, Weta used mocap and Simulcam to project the film’s actors into the digital jungle environment of Pandora, allowing Cameron to see the composite image in real-time and direct accordingly.

More recently, the immensely popular Star Wars series The Mandalorian, has been the go-to example of how virtual production is innovating Hollywood. The Mandalorian production team and effects artists at Industrial Light & Magic used a largescale LED video wall to produce the show’s digital environments behind actors as they performed. This allowed the camera to capture the live performance and digital backgrounds simultaneously during principal photography.

The Benefits Of Virtual Production

In both of the above cases–and in many others–virtual production presents media makers with new approaches to their craft. In addition to impressive end results, virtual production offers benefits across all stages of production–from development through post. The ability to visualize what was once post-production VFX work in production or even preproduction means more time to refine these effects. It means directors can make more informed decisions during principal photography because they can actually see what they’re working with. Actors and directors alike have long bemoaned the traditional greenscreen process as alienating and unnatural. Particularly with these most recent developments in virtual production, everyone is given the ability to see digital elements in real-time, which creates a more natural and engaging approach to performance than was previously possible for effects-heavy projects.

A Dollar Saved Is A Dollar Earned

In addition to creative benefits, the collapsing of production phases through virtual production streamlines workflows. This means time saved and–more importantly–money saved, which in commercial production industries is perhaps the greatest motivator behind the continued adoption of virtual production.

Will IKIN Ever Play A Role In Virtual Production?

Short answer: only time will tell. Currently, IKIN is focusing its efforts on refining its holographic technology, building out industry partnerships, and preparing its dimensional platforms for special industry use cases and an upcoming consumer launch. That both the IKIN ARC and RYZ holographic systems are Unity-based, however, opens the door to the possibility of IKIN involvement in virtual production at some point down the line. Unity’s recent acquisition of leading visual effects company Weta Digital, with its deep cache of proprietary virtual production tools, signals that Unity is assuredly moving up in this space. In the years to come, look for Unity to continue building up their Unity real-time engine while expanding their footprint in the greater realm of production, virtual in particular.

To learn more about IKIN’s ongoing developments in holographic technology and hologram applications, please visit the IKIN blog. And, remember to follow IKIN on social media.