Bringing VFX to Life in the Physical World with Virtual Production
DNEG uses pixel-perfect motion tracking of its cameras to support filmmakers’ creativity
Headquartered in London and with studios across the globe, DNEG is one of the world’s leading visual effects and animation companies, boasting credits including Avengers: Endgame, Westworld and Tenet, along with five Academy Awards. While its services are wide-ranging, its Vicon motion tracking system has one very specific function – to help filmmakers to develop their vision using virtual production techniques.
Virtual production effectively enables directors to visualise their VFX in real-time rather than having to wait for sequences to be developed and rendered. Some form of tracking technology is attached to a ‘virtual camera’ which the user can then move around a physical set, merging physical actors or objects with digital effects on a screen. The technique enables directors and cinematographers to get a sense of what their finished effects will look like or to compose shots.
Isaac Partouche, DNEG’s Head of Virtual Production, saw the value of those techniques early. Prior to his work at DNEG, Isaac established his own VFX company in 2007, SolidAnim, and (among other things) developed with his R&D team a piece of software called SolidTrack, based on SLAM algorithms. The tool enabled directors to visualise an animation in three-dimensional space.
“SolidTrack was a mixed-reality tool created to replace, in real-time, blue or green screens on-set. It would detect features, corners, edges, etc… and replace the screens with on-set previs” says Isaac. “We provided filmmakers with a representation of what the final image was going to be. However, at the time, the ‘footage’ seen through the virtual camera didn’t match the quality of the final visual effects.”
Isaac wanted greater control of the workflow and the ability to transition from rough virtual camera pre-vis to the final effects more seamlessly. To do that, he needed incredibly precise motion tracking of on-set cameras.
Fast forward to 2020 and Isaac, now working for DNEG, has closed the gap with pixel-perfect positioning of cameras that enables him and his colleagues to seamlessly merge footage with motion capture and other VFX data.
Adopting the right tracking technology was crucial. He could have opted for markerless optical tracking or mechanical encoders which attach to cameras, cranes and dollies. The former, however, lacked the accuracy and framerate he needed while the latter is too expensive and time consuming to set up.
DNEG’s in-house virtual production lab in London uses a Vicon setup of 24 cameras coupled with Tracker and Shogun software, giving the team the accuracy that it needs, along with the flexibility to expand or reduce the setup as needed.
“It makes sense to have the same form of tracking for both the actors and the camera; it is easier to combine the data”, says Isaac.
“We need to synchronize everything: the camera speed, the motion capture, the LED panels, etc… Everything on set has to be on the same frame rate. Everything has to line up, down to the level of a pixel or, in some cases, even smaller.”
Isaac also praises Vicon’s Tracker software, noting that its anti-jitter features mean that even on a busy set where markers might get jostled or occluded, the data remains consistent.
THE GROWTH OF VIRTUAL PRODUCTION
Virtual production has exploded in importance in recent years, with more and more VFX studios making use of the technique.
More than just a way of speeding up production, it has become an important tool that boosts filmmakers’ creativity.
“DNEG’s approach is to utilize cutting-edge technology in pursuit of our clients’ storytelling goals,” says Isaac. “The director, the director of photography and the crew can visualize the work in real-time in order to explore their vision further.”
DNEG’s Virtual Production crew has delivered services for productions such as Denis Villeneuve’s ‘Dune’ and Kenneth Branagh’s ‘Death on the Nile’. It is currently in high demand to help content creators for both film and episodic projects overcome the impact of the COVID-19 pandemic on production. By leveraging advances in computing power, DNEG’s VFX expertise and game-engine technology, DNEG Virtual Production offers filmmakers a range of solutions to keep the production going while respecting social distancing and other safety measures in place to protect the
cast and crew.
“Virtual Production is also a useful tool for engaging directors who previously might have eschewed visual effects because they felt disconnected from them,” adds Isaac. “All our tools at DNEG are designed to fit within our clients’ workflows and processes. We want to make things as simple as possible for the filmmakers and provide a creative setting that will support their storytelling.”
With more powerful virtual production tools, DNEG is able to support the creativity of filmmakers at every level, from directors of multi-billion dollar blockbusters to auteurs creating most intimate character films.
