Our SIGGRAPH markerless showcase demonstrated that this technology is finally ready for the mainstream

In the world of technology, most of the time you can be first or you can be best.

Markerless motion capture has been around in one form or another since the 1990s. Occasionally someone has produced a version of it that has been technically impressive. But technically impressive is not the same as fit-for-purpose.

Five years ago I met with our CEO, Imogen Moorhouse, to survey the state of the art. We concluded that the technologies underpinning markerless had matured sufficiently for Vicon to build a truly robust, powerful, markerless motion capture solution.

Half a decade later at SIGGRAPH 2023, our markerless showcase, developed in partnership with Dreamscape and Artanim, proved that we were right.

It took a lot of work to get there, though. Before we even started, we needed certain extrinsic factors to come together. Without the development of machine learning and AI that’s taken place in the autonomous vehicle field, or the raw computing power that has become available thanks to advances in GPUs, markerless would still be in the dark ages.

Then we needed to build a team. When I started as CTO at Vicon, there was no such thing as the markerless technology team—so we built one from scratch. We needed specialists with skill sets that didn’t exist a decade ago, such as machine-learning scientists, AI data analysts and ML ops personnel, not to mention all the infrastructure that comes with doubling your team.

And then there was the invaluable input of the Vicon community. We established the Pioneer Program to invite our customers to help shape the future of this important technology. You responded in droves.

There are now over 200 organizations signed up from across the breadth of our customer base. We have everyone from location-based VR companies, such as Dreamscape, to AAA VFX studios, to indie games developers. We’re thrilled, in particular, by the interest from our life sciences community—researchers from hospitals, universities and sports organizations have signed up to investigate our markerless solution and its potential applications.

You’ve shown us exactly what we wanted—a host of fascinating use cases, many of which we never would have considered if we were working in a silo.

VFX users, for example, are keen to slot markerless into their pipelines early. While the mocap stages for blockbuster films will always require the extreme precision of markered optical tracking, our VFX users have shown us that they can reimagine their shoots by including markerless in their pipelines, either by integrating it with optical or in the form of markerless pre-vis.

Dreamscape and Artanim, of course, have demonstrated how freeing VR users from the need for markers can give them an incredible, immersive experience. (For more on the resounding success of our SIGGRAPH showcase, see our accompanying article.)

The life sciences community has told us that without the need for marker placement, they’ll be able to capture the movement of patients for whom it wouldn’t be possible otherwise, such as children or the elderly. Similarly, using markerless makes capturing children and people with mobility difficulties smoother and faster, enabling clinicians to see more patients in a compressed period of time.

The strategy that underpins all of these use cases comes back to accessibility every time. Being able to walk into a volume with a minimal array of cameras and just start shooting creates a huge array of options. Users without extensive backgrounds in motion analysis can tap into the tools with much greater ease. Suddenly, motion capture becomes accessible in circumstances where previously it wouldn’t have been. 

Importantly, though, the Vicon markerless solution is backed by our deep expertise in motion capture. It’s an end-to-end markerless solution that fits within your existing pipelines—we’re not asking you to learn a new tool and process. And because this will be a Vicon solution running on Vicon hardware and integrating with Vicon software, the end result is a fully integrated solution backed by Vicon Support from start to finish.

Furthermore, our solution doesn’t put you in an either/or situation where you have to choose one mode of motion capture over another. You can mix and match tools, even within the same system, to get the results you need. Dreamscape’s Clockwork Forest adventure at SIGGRAPH was a perfect example of a multi-modal solution—users’ bodies were tracked using markerless to create a sense of immersion and streamline the experience, while the VR helmets and props were tracked using active markers.

We know that we’re only scratching the surface here. One of the most exciting aspects of working at Vicon is seeing our customers come up with uses for our technology we never dreamed of. 10 years ago, no-one thought that Vicon solutions would be used to manage automated drone cities, for DIY real-time content creation, or for performing virtual maintenance on aircraft. We’re excited to see where our customers take our markerless technology once we release it to the world.

I’ve been Vicon’s CTO for five years now. Every one of those years has felt like another leap forward for my team and for this company. A huge part of the work, for me, has been developing our markerless solution and the infrastructure we need to deliver it. And we’re ready. My sixth year as CTO will not only be about the regular cadence of releases for our existing solutions—it will also be about executing on all the preparation and building we’ve been doing in the markerless space.

2024 is the year that we release our markerless motion capture solution to the wider Vicon community and see the amazing things you’ll do with it.

The debut of Vicon’s markerless motion capture solution at SIGGRAPH 2023 brought to life an impressive, immersive experience that dazzled attendees. The showcase featured Clockwork Forest, a new virtual reality adventure built by Dreamscape that shrinks participants down to the size of an ant to explore a magical, super-sized world.

“I’ve been coming to SIGGRAPH since 1996 and this is the best thing I’ve ever seen here, hands down,” said one delegate. “I think this technology is going to revolutionize a lot of theme park experiences, and I’d love to see what it could do for education and museums where you could experience being part of historical events. I think it has a lot of applications in training, too. It was just breathtaking.”

As well as winning plaudits from delegates, Vicon’s markerless solution was recognized by CGW (Computer Graphics World) with the Silver Edge award as one of ‘the most innovative and impressive technologies’ at SIGGRAPH.

Six users at a time were able to explore the Clockwork Forest’s rich, interactive environment wearing only a VR headset, thanks to the multi-modal, real-time solving of Vicon’s markerless technology.

Dreamscape, driven by its research arm, Artanim, develops immersive VR storytelling experiences for entertainment, training and education. The capabilities it demonstrated at SIGGRAPH mark a leap forward in location-based VR, reducing friction for users and increasing their sense of immersion in the virtual environment.

“What we showcased is the ultimate evolution of our virtual reality solution,” said Sylvain Chagué, Founder and Chief Technology Officer for Artanim and Dreamscape. “Thanks to the markerless tracking solution and the cloud streaming of VR experiences, we’re able to reduce the equipment to just a head-mounted display and offer the same quality of immersive experience as we used to with a fully markered, optical tracking system, so this is incredibly exciting.”

President and COO of Dreamscape, Aaron Grosky, added, “We have been anxiously awaiting the time when markerless could break from concept and into product, where the technology could support the precision required to realize its amazing potential.

“Vicon’s reputation for delivering the highest standards of motion capture technology for 40 years and Dreamscape’s persistent quest to bring the audience into the experience with full agency and no friction meant that working together on this was a no-brainer,” Grosky expanded. “We’re thrilled with the result. The implications for both quality of experience and ease of operations across all our efforts, from location-based entertainment to transforming the educational journey with Dreamscape Learn, is just game-changing.”

For more on Artanim and Dreamscape, visit https://dreamscapeimmersive.com/ and https://artanim.ch/.