Crossing the Divide Between VFX and Life Sciences

Formulating a name for Luke Hopper’s role at the Western Australian Academy of Performing Arts’ motion capture stage is difficult. “I haven’t thought of a catchy title for myself yet,” he jokes. His role has become increasingly awkward to pin down because the work he does now often traverses motion capture disciplines and their different languages.

“It’s been interesting, crossing between the languages that the entertainment guys and the life sciences guys use,” he says. “It’s the same dots and the same cameras, but you have to switch your brain into life sciences or entertainment. It’s this funny sort of competitive collegiality.”

Luke’s letterhead avoids putting him into either camp, giving him the vague title of ‘Vice Chancellor’s Research Fellow’ at Edith Cowan University. A biomechanist with extensive experience in ballet by training, Luke now works with the university’s motion capture lab on both entertainment and life sciences projects, increasingly bringing the two together.

A MOCAP MELTING POT

That crossover isn’t an accident – it’s part of the lab’s DNA. “We thought it would be an interesting experiment to see what would happen if we brought scientists into the Arts Academy,” Luke says.

The experiment resulted in a capture volume equipped with 12 Vicon T40 cameras and, following a research trip by Luke to the UK, six Vicon Vantage 8 cameras along with Nexus, Blade and Shogun software.

That research trip, funded by a Churchill Fellowship, proved particularly instructive for the capture studio’s current direction. Luke toured major dance and performing arts research centres as well as visiting leading companies in the motion capture field including the likes of Audio Motion and Vicon. “It made me realize you can do really excellent, world-class work with just a handful of people. You just have to contextualize it within the scope of the way the studio was going to operate,” Luke says.

“In Australia, we’ve got a lot of opportunity in that we’re a Performing Arts Academy, we’ve got animation and game design, film and TV all on campus with the Motion Capture Studio. So there’s some great potential for that interaction and collaboration happening there.”

Despite Luke’s background in biomechanics, he’s been surprised by how entertainment applications have come to the fore. “We’ve found some good collaborations with the health school and within the arts and humanities school and in game design,” he says. “So I’m increasingly working in the visualization space and using the motion capture for production of aesthetic outputs. We’ve still a couple of biomechanical projects on the go, but I really feel like the direction is changing a lot.”

LIFE-SAVING APPLICATIONS

One particular project that encapsulates the cross-discipline approach of the lab is a training simulation for paramedics. “It’s for them to practice triage of multiple casualties and prioritization of casualties in a mass trauma situation,” Luke says.

“Normally it’s really expensive to run. Typically you’ll have to employ 10 to 20 actors, you have to give them the brief, you have to have a big scenario set up and then you get maybe two paramedics through an hour.” With the virtual version based on mocapped actors, Luke says, it’s possible to get many more trainees through a simulation – something they might not otherwise have done before graduating.

“It’s been a really great experience because no one person within the project could have done it by themselves,” says Luke.

“Knowledge of the health sector was just invaluable in providing that rigorous information so that we’re producing something that’s relevant and that the paramedics aren’t going to look at and go, ‘well that’s not right’. But they’ve got very little experience in game design and all the mechanics and the programming that you need to work that knowledge into the application.

“Nor did the game design guys have a good handle on animation and motion capture. So it was really quite fortunate that we got people that could all speak to each other and work together in that space. It’s been fun in that way, we’re continually trying to translate one another’s intentions into our own disciplines.”

Another project has directly brought life sciences data processing techniques into the field of animation. “Working with a Japanese colleague we’ve published a paper that basically uses biomechanical methods and marker placement and algorithms in the Blade animation software, so that you’re producing a scaled avatar that matches the body proportions of the actor, so We’ve been using that in dance because it’s a sort of a first step in bypassing a lot of the assumptions that are made in the animation models for the fast production,” says Luke.

“One of the software developers in Oxford said to me it’s probably a good way to go in that the labeling and the processing in the animation software is actually more powerful because of the developments that have happened in that space. So when we’re doing a biomechanical application we’re sort of crossing over and in between the two,” he adds.

Luke stresses the fact that there’s still a lot of work to do when it comes to bringing his lab’s practices in VFX and life sciences together. “But we’re really hoping that we’ll start to get some projects off the ground that start to push the two together,” he says. “Let’s see where it can go from there.”