Fusing Marker and Markerless Motion Capture

The idea that motion capture requires creators to choose between either fidelity or speed is becoming a thing of the past. Using markers delivers sub-millimetre accuracy and rock‑solid solving skeletons for performers and props, especially for action with complex contact points. Markerless mocap gives actors and directors freedom to move freely, quickly, and capture nuance on the fly. Hybrid motion capture combines both – not just simultaneous data capture, but fusing them to leverage their unique advantages – so you can lean on markers for precision and realism while using markerless motion capture to broaden coverage, accelerate blocking, and keep talent in the creative flow.

At SIGGRAPH 2025, we showed a practical, creator‑first workflow that blends the speed of Vicon Markerless with the fidelity of markered, so that you can direct, iterate and lock detail without changing environments. If you missed it, catch the highlights and resources on the SIGGRAPH hub or watch the video on Fusing Marker and Markerless below.

Marker‑based motion capture uses Vicon cameras to track small reflective markers and reconstruct a performer’s movement with exceptional spatial fidelity. It shines when you need the highest levels of data accuracy – repeatable alignment, confident contact moments, and consistent retargeting across sequences, including complex prop work and multi‑character interaction.

Markerless motion capture applies multi‑view video and machine‑learning models to solve a performer’s pose without suits or markers. It accelerates setup, and lets directors explore blocking and intent at the speed of rehearsal – perfect for previs, iterative look‑dev and wide scene coverage. Engaging gameplay relies on player interaction. This is why markerless frees up ideation for animation prototyping, such as emotes, physically challenging combat sequences or general player interactions that could have numerous variations – all scenarios that can be limited by exclusively using markered mocap.

Sometimes, scenes or actions call for the speed of markerless but with marker-level accuracy on key parts of the performance. Perhaps they’re attaching a virtual camera to a performer’s head for a first-person cinematic perspective, or their hands are interacting with precisely arranged scene dressing. Used together, markered motion capture anchors the beats that must be exact, while markerless expands coverage and keeps performances flowing. The result is a faster, more flexible stage that serves film, games and VR from first pass to final shots.

These steps run through the basic process as demonstrated in the video, and show how quickly and efficiently this can be done:

1) Calibrate the performer in Markerless. The actor steps into the volume and is calibrated as a tracking subject by running a quick calibration (including a T‑pose), and immediately streams a stable skeletal solve that’s excellent for broad motion and fast exploration.

2) Add lightweight optical detail where you want more articulation. Place small marker clusters on any combination of hands, head or feet, and register them as simple “props” (Head, Left Hand, Right Hand). There’s no special layout; the goal is to elevate fidelity only where you want extra nuance.

3) Capture a short range‑of‑motion clip. With subtle wrist rotations, head turns, and small articulations recorded, it’s put into Shōgun Post – the same Post used for optical – and run a short script that associates those prop tracks with the performer’s skeleton. The result of this is an updated VSK (Vicon Skeleton), which fuses the markerless subject with the markered clusters.

4) Round‑trip back to real time. Export the updated VSK and load it into the Vicon Markerless real‑time app. The UI now shows plus icons for the enhanced joints, and you assign the live prop tracks to those placeholders. From that moment, the stream carries the ease and coverage of Markerless with optical fidelity on the targeted areas.

5) Scale it to your scene. The same method can be used for feet or any limb that benefits from higher articulation. You keep wardrobe‑true performances and rapid blocking, while precise movements, like head orientation and hand nuance, arrive with optical‑comparable quality.

This hybrid approach lets film, games and VR teams move quickly in rehearsal and previs, then lock critical detail without re‑engineering the stage. As demonstrated live, the workflow runs in real time and will be delivered in Shōgun later this year.

Most importantly, hybrid puts the focus back on performance and direction: faster iteration, clearer decisions, and a stage that supports storytelling from first pass to final shot.

Accuracy where it counts

Use markers on bodies or props that demand tight physical alignment – think weapon arcs, hand‑to‑object contacts, or precise foot placements – while running markerless mocap across the full scene to maintain continuity and coverage.

Speed where it matters

For quick mocap performance exploration, run markerless passes to block beats, explore the camera, and test timing. When shots lock, add markers selectively to nail critical moments without re‑engineering the entire setup.

Resilience to real‑world chaos

Markerless helps when markers occlude or drop; markers help when camera angles, lighting shifts, or crowding challenge a purely markerless solving skeleton. The combination increases the percentage of takes that are immediately usable.

Cleaner retargeting and less cleanup

Hybrid data stabilises joints that are traditionally tricky in high‑energy moves, reducing solve jitter and the downstream keyframe massage that slows teams between dailies.

Studios are already blending approaches to move faster in virtual production and games.

Dimension streamlined virtual production by integrating Vicon’s markerless technology into their workflow, enabling quicker creative decisions while preserving fidelity where needed.

Framestore accelerated reviews by using Vicon’s markerless technology to iterate earlier in the process, helping directors and animators make confident choices sooner.

Film & episodic TV

Dialogue‑led scenes benefit from the ease of markerless for wardrobe‑true performances and fluid blocking, with selective markers added for signature movements, poses and interactions that must land precisely. The outcome is more takes per hour and performances that read beautifully on screen.

Games

The hybrid approach gives creators choice – markerless provides a lightweight option to get increased data quality for instances where it’s needed, without having to incur the expense and complexity of a full markered system. This means that teams can deliver engine‑ready clips sooner, with greater consistency across sets.