AMGI wants to make Disney-quality animation in real-time

In the world of VFX, motion capture is conventionally viewed as a tool that’s best suited for making live action film and TV. AMGI plans to turn that perception on its head by using digital puppeteering to drastically reduce production times in animation.

“I always joke that we try to be the SpaceX of animation,” says Colin Brady, AMGI’s Chief Creative Officer. “We want to just break whatever the system is, we want to fail as fast as possible so that we get our mistakes out of the way. We want to try something new every day.”

Brady cut his teeth as an animator on the first Toy Story film for Pixar, going on to work closely with Steve Jobs and teams at Lucasfilm and Jim Henson’s Creature Shop.

“In the early days I always found myself acting everything out with my body, using my video camera. I would sneak off to the bathroom because it had the largest mirror and I would plan out my shot. Even back then I thought, ‘There’s got to be a better way to do this,’” Brady says.

Brady’s solution was motion capture, though the idea is not without its detractors. “They’d say, ‘Motion capture, that’s cheating’. Or, ‘That’s not for animation. That’s more for live action’. And I think nothing could be further from the truth.”

He found kindred spirits when he met Luke and Roger Paglia, now COO and CEO of AMGI, and the trio began the work of disrupting the way animation is created.

DIGITAL PUPPETEERING

“What we’re doing that’s unique is focusing on puppeteering with non-human characters, retargeting human proportions to cartoon character proportions,” says Brady. The process uses Vicon-powered motion capture mixed with keyframe animation and processed using the render speed of Unreal Engine. The result is animated characters puppeteered by tracked human actors, dramatically reducing production times.

The bedrock of the process is AMGI’s Vicon setup. “We have a beautiful stage that’s about 20 ft by 25 ft with 16 Vero cameras,” says Brady.
“They’re great, we love the precision and they keep the calibration really well. I think it’s my favorite stage I’ve ever worked with.

“Vicon, working hand in hand with some smart programming, really gets this process to stick together. If there are two lungs in the body of what we’re doing, they’re Vicon and Unreal.”

“What’s so great about this system,” adds Luke Paglia, “is that you can put markers on anything and if you translate the data accurately in a creative way, you can do so much with it.”

PUTTING THE SYSTEM TO WORK

AMGI has used this process to do work-for-hire for the likes of Coldplay and to rapidly produce a portfolio of its own IP, including a property coming soon to Netflix as a feature film.

AMGI also has an app, set to launch in 2022, that uses phone cameras to capture facial movements and then animates characters based on the data it produces. “Our goal is to create Disney-style, high quality characters that can work on an iPhone,” says Brady. “Essentially we pre-record a lot of motion using dancers on stage and we trigger that in a very smart way.
I think we have the highest quality animation that’s ever been put on an iPhone. We think it could be the TikTok of animation apps.”

While the technology is interesting in and of itself, it’s the way that AMGI sees it impacting the animation production pipeline and facilitates creativity that really excites Brady. “Going back to Pixar days, we were lucky if we could animate four seconds a week by hand,” he says. “I’ve worked on shots for three weeks that were less than one second. I always look at that as a benchmark. Probably the largest portion of an animated feature film budget is how many seconds can be produced per week, per person. If you can turn those four seconds into 10 seconds, to 20 seconds, a minute, that changes everything.

“For live animation, a live TV show or an improv show, an animator could potentially do four minutes a week, and then you’re animating an entire show in a week with five animators. It’s crazy.”

AMGI even envisages using the process to sidestep platform holders such as the streaming services to reach audiences directly. “You could do a live show that streams on YouTube and reach 100 million says Brady. “So we’re really starting to get excited about bringing actors on stage and essentially doing live, SNL-style improv sketch comedy with our puppeteered characters.”

HUMAN HURDLES

For all the technical challenges that AMGI has already overcome, it’s the misperception of motion capture in the animation world that remains the sticking point.

“The biggest hurdle, to be honest, is that motion capture has always been viewed as an icky technical thing by most traditional animators,” says Brady. “What it really takes is for the artists to embrace this. The scientists will never win by pushing this technology without the artists. But once animators actually get on stage themselves and they try it out, they get it. They see the efficiencies. They’re hooked.”

Internally, AMGI already has an experimental culture that encourages animators to follow their curiosity. “We treat it like a big playground where, if you want to pick up a box and you want to start playing around with how you can capture and animate it, go play around! What’s beautiful about it is the motion is high quality enough to be used for final production, but it’s also inexpensive enough to experiment with,” says Brady.

For all the technical wizardry behind AMGI’s production pipeline, everything comes back to that playful approach. “Ultimately, what we’re creating is memorable characters. All this is in the service of storytelling. What’s exciting is that the more efficient you make the process, the more courageous you can be with the stories you tell. You don’t have to wait for a studio to greenlight your show. You could say, ‘You know what, let’s all come in on a Saturday and just do the movie.’”

Looking ahead, Brady has his eye on extended reality. “Of course, people talk about the metaverse. And yes, we’re very interested in that. There’s a lot of excitement there.”

It’s telling stories that comes first, however. “Ultimately, I do think narrative is what drives culture,” says Brady. “I’m very excited about live cartoon animation, because I feel that we relate to it in a very basic way. I really do have a goal of making nothing less than Pixar- or Disney quality animation that’s done live, that no-one knows is motion capture. And I think we’re getting there pretty rapidly.”