Digital Humans Unite!

The dust from the amazing FMX 2018 conference has settled, the Vicon VFX booth has now been packed away until SIGGRAPH, and the team are recovering from Fruehlingsfest (http://www.stuttgarter-fruehlingsfest.de/en/home/). It’s time to reflect on what happened on the booth.

We had two amazing show pieces taking place this year: #MeetSiren and Vicon’s very own escape room, #VEX.

#MeetSiren

Every day, the Siren performance (played by the amazingly talented actress Alexa Lee and MC’d by Vicon’s product manager Tim Doubleday and EPIC’s Ben Lumsden), was met with great crowds at each demo. So, what were the crowd seeing from Vicon? Siren’s body and finger motion was being tracked and solved in real time and streamed into Unreal Engine. Sh, where we were using a ground-breaking five-finger-solving algorithm. We really wanted to see those fingers!

Streaming into game engines is not a new thing, however what was new and being shown at FMX (and previously at GDC) was Shōgun streaming directly into Epic’s Unreal Engine via the new LiveLink plugin, Vicon being the only optical mocap vendor to be currently supported – thank you EPIC!

While Vicon delivered an unbreakable real-time finger and body solve, what really brought Siren to life was the amazing real time facial performance that was delivered by Cubic Motion’s unique facial pipeline, which drove the cutting edge facial rig delivered by 3lateral. The combination of Vicon, CubicMotion, 3lateral and EPIC was a dream collaboration and really helped to elevate our presence at FMX.

On Thursday and Friday at the festival, we decided to do the demo live into the digital human panel show while Cubic’s Steve Caulkin was on stage. We also streamed Siren around the globe via YouTube so anyone could view the demo. I think we all agree that Siren delivered a top performance!

A highlight of #MeetSiren, and one that produced possibly the biggest round of applause of the week, occurred when Siren went off script and had Vicon’s own Tim Doubleday dancing on the booth to Black Betty.

If you haven’t seen Siren at either FMX or GDC, please watch the making of video and live performances and you’ll see why this was not your average motion capture session. We’ll have a more in-depth tech breakdown coming soon.

#VEX

Once again, the VEX escape room experience had queues of players wanting to work with our captain Riley and figure out the puzzles to escape back to Mars!

What made this VR experience unique was that the player not only got a full body immersive experience via a Shōgun quick marker set, and finger immersion via Manus VR data gloves, but could chat to a real-time digital avatar, played by the amazing Gina Williamson, who had her body data solved by Shōgun and facial animations driven by a Dynamixyz head rig and real-time facial solver. Gina, playing the captain, took players on a journey from a marooned spaceship that was in desperate need of repairs. The players carried out the repairs by solving the puzzles to bring parts of the ship back online and make it back down to Mars.

Inside the virtual escape room, the puzzles may look easy, but get deep inside the game and they try even the most hardened of puzzle players. Adding another dimension to the experience by making the players catch a torch that was a prop thrown by the host in the virtual environment, validated the accuracy of the solving capabilities.

With a constant flow of willing players, the reduced marker set and Shōgun software made switching between players a much quicker process than was previously possible with a normal full marker set and suit.

Thank you to all the visitors and participants on the Vicon booth at FMX and we look forward to seeing you again next year with a new demonstration.

PS. If you haven’t been before, be sure to go and experience the Fruehlingsfest that takes place at the same time as FMX. You might just find a table with a few familiar faces on dancing with our unicorn!!