The crowd is the work of Pixomondo (‘PXO’), produced in collaboration with Alter Ego, William F. White International and PXO’s Virtual Production Academy. It answers a long-running problem for advertising. “One of the problems that we face in commercials all the time is trying to shoot large-scale crowds in a large environment,” explains Eric Whipp, cinematographer on the ad. “First of all, getting into a stadium is very tricky, filling it with a crowd is very tricky.”
“As any assistant director will tell you, wrangling 25 fans as extras is a pretty tall order, but wrangling 20,000 fans would be impossible and extremely costly,” expands Matthew Manhire, co-director of the advert. “Pixomondo had this fantastic idea that they were testing with, and as a result we’ve got 20,000 unique individual fans doing various random movements to make the stadium have a sense of life and a sense of humanity.”
Traditionally, that kind of crowd would be done using a couple of rows of real extras in front of a green screen, with the remainder added in post-production. It’s a costly, challenging and time-consuming process that Pixomondo had gone through recently for HBO’s Winning Time: The Rise of the Lakers Dynasty.
In the wake of Winning Time the studio began working on a real-time solution that would achieve the same results in a way that was more cost-effective and offered more creative flexibility. In-camera effects using LED walls usually follow the lead of The Mandalorian, offering epic environments that are typically deserted. PXO set about building a system for filling those environments with people.
The crowd was developed using Vicon-powered motion capture combined with 3D scans of real people to create the animations. The system was then put together in Unreal Engine, with users able to trigger different crowd emotions that deploy over 100 different animations for any given subject. The crowd can be displayed on the LED wall of a virtual production volume, with Vicon technology used to track camera position in relation to the imagery being displayed. The Caledon FC commercial was the perfect opportunity to test the new system out.
The importance of crowds
“Improving the capability of crowd assets is important to PXO and the virtual production industry, because it proves that we’re not relegated to certain kinds of abstract assets; that we can accomplish any asset that requires a crowd,” says Christopher Cox, virtual production producer for PXO.
The system also brings creative advantages to a shoot. “The creative decision maker gets to choose on the day what level of detail they want to adjust, therefore getting more out of the day,” Cox says. “Sometimes, shutting down a set to change costumes or to change out patterns that are seen in the crowd can really reduce the efficiency of a shoot, whereas with virtual production those decisions are clicks of a button to swap out variables, and you can remain efficient.”
“We bring the athlete in, we get what we need and then we’re done, rather than putting them into an unpredictable environment, whether it be unpredictable weather or the unpredictable talent in the background,” Manhire elaborates. “We have full control of everything, so it’s a matter of plunking them in and them doing what we’ve been rehearsing in our prep.”
“We have the luxury of not having to have the huge cost of renting a stadium,” adds David Whiteson, the commercial’s other co-director. “If we were renting a stadium, not only do we have the people problem, we have the cost of renting it for two days. Here, we can do more shots in a day because we’re not fighting the sun and we can be a little bit more creative.”
Shooting at high speed
Creating a real-time, reactive crowd wasn’t the only technical achievement of the shoot.
“A very big thing is we’re shooting at 200 frames per second,” says Whiteson. “When you do that, it causes a huge amount of flicker on the LED wall because it’s not in sync with the camera or the lighting. With this project we have found ways around that to achieve a high action scissor kick in slow motion that’s never been done before. That just adds to our list of things that we’ve been able to develop and achieve to progress this medium further and further.”
Another technical challenge – something doubly important for filming soccer, in which most of the action happens at ground level – was making the grass seamless. “Typically, up until now, virtual production has been about mid-shin up because we’ve wanted to hide the seam where the screen meets the floor. But through the discoveries that we’ve made with Pixomondo
and Alter Ego we’ve found ways to properly blend the floor and the screen,” says Manhire.
“Pixomondo obviously made massive developments on their shows like Star Trek, so we brought that over into the commercial world and we really wanted to highlight that. So we’re seeing a full interaction of soccer ball, cleats and feet, and soccer players moving across the turf, blending seamlessly with the digital environment behind it,” Manhire adds.
The meeting point between floor and screen might not seem that important to the layperson, but for Cox it’s the point at which the entire project comes together. “Blending the virtual with the practical lighting can be really tough, but the most important part of the day is making sure that the floor is finding its way into a virtual blend so that the scale of the asset can actually be felt,” he concludes.