What Is Teleoperation? How Humans Control and Train Robots 

Robots are getting more capable, but most still can’t handle an unfamiliar task on their own. Teleoperation fills that gap. A human takes control, the robot does the work, and every movement can be recorded. That makes teleoperation in robotics one of the main ways humanoid robots are being trained today. 

Teleoperation is the remote control of a robot or machine by a human operator, who sends commands from a distance and receives feedback about what the robot is doing. The word combines “tele” (at a distance) with “operation”.

So what is a teleoperated robot? It is any robot whose movements are directed, fully or partly, by a person in real time rather than by its own software alone. The operator might be in the next room, on another continent, or on Earth steering a rover on Mars.

Teleoperation works as a continuous loop: the human operator sends commands, the robot carries them out, and sensors send feedback so the operator can adjust. Every human teleoperation setup has four parts: 

1. The operator interface. A joystick, hand controller, exoskeleton, VR headset, or motion capture system that turns the operator’s intent into commands. 
2. The communication link. A cable, wireless network, or satellite connection that carries commands out and data back.
3. The robot. Its controller converts incoming commands into joint movements, grip forces, or wheel speeds. 
4. The feedback channel. Cameras, force sensors, and joint encoders show the operator what happened through video, haptic feedback, or both. 

The quality of that loop decides how well teleoperation works. Latency, the delay between a command and its result, is the biggest factor. A few hundred milliseconds is enough to make fine manipulation feel clumsy. 

Teleoperation systems differ mainly in how much control the human keeps and how their movements are captured. These are the four main types. 

1. Direct control 

In direct control, the operator commands every movement, and the robot has no autonomy. Joysticks and leader-follower arms, where a follower robot copies a hand-held leader arm, are typical. It is simple and predictable, but tiring for the operator and sensitive to latency. 

2. Shared control 

In shared control, the human sets the goal, and the robot handles the low-level details. The operator might point to an object while the robot plans the grasp and avoids collisions. Supervisory control goes further: the operator issues high-level commands and the robot completes each step on its own. Mars rovers work this way, because signals take several minutes to travel between Earth and Mars. 

3. VR teleoperation 

VR teleoperation puts the operator inside the robot’s view, using a headset and tracked hand controllers. Stereo cameras on the robot stream video to the headset, and hand movements map to the robot’s arms. It feels intuitive, but consumer VR tracking is limited to the head and hands, and accuracy can drift. 

4. Motion capture-based teleoperation 

Motion capture-based teleoperation tracks the operator’s body, hands, or fingers with an optical system and maps those movements onto the robot in real time. It captures far more of the body than VR controllers, at high frame rates and with sub-millimeter accuracy. That makes it suited to dexterous tasks and to recording clean training data, as covered below. 

Teleoperated robots are used wherever a task is too dangerous, too distant or too delicate for a person to do directly. Common examples of teleoperated robots include: 

• Surgical robots. Systems such as the da Vinci let surgeons control instruments through small incisions from a console. 

• Bomb disposal robots. Tracked robots let operators inspect and disarm devices from a safe distance. 

• Underwater ROVs. Remotely operated vehicles inspect pipelines, cables and shipwrecks at depths divers cannot reach. 

• Nuclear and hazardous-site robots. Teleoperated machines handle radioactive material and inspect damaged facilities, such as Fukushima Daiichi. 

• Planetary rovers. NASA’s Mars rovers are driven through supervisory teleoperation from Earth. 

• Humanoid robots. A growing number of humanoids are teleoperated, both to perform tasks and to collect training data. 

Humanoid robot teleoperation is the real-time control of a human-shaped robot by mapping a human operator’s body, arm, and hand movements directly onto it. Because a humanoid robot shares our proportions, the operator can move naturally, and the robot follows. 

That mapping is called motion retargeting. Human and robot limbs never match exactly, so software translates joint angles, scales distances, and respects the robot’s joint limits. Hands are the hardest part. A human hand has more than 20 degrees of freedom, and dexterous robot hands need finger-level tracking to grasp, pinch, and manipulate objects. 

Humanoid teleoperation is not only about remote control. It has become one of the fastest ways to teach humanoids new skills. 

Every teleoperated task produces a demonstration: a synchronized record of what the robot saw, what the human commanded, and how the robot moved. Collect enough of these demonstrations and a robot can learn to repeat the task on its own. 

This approach is called imitation learning, or learning from demonstration. It sits at the heart of embodied AI, where robots learn through physical interaction rather than from text or images alone. Projects such as Stanford’s ALOHA showed that low-cost bimanual teleoperation could teach robots fine tasks from only dozens of demonstrations. 

The catch is data quality. A policy can only be as good as the demonstrations it learns from. Jittery tracking, dropped frames, or poorly synchronized data teach the robot the wrong movement. That is why teams building physical AI increasingly pair teleoperation with precise motion capture, which also serves as ground-truth data to check what the robot actually did. 

Optical motion capture gives teleoperation the precise, low-latency tracking it needs to control the robot in real time and record clean training data. It tracks the operator’s whole body, hands and fingers, plus the robot itself, in one shared coordinate space. 

Vicon Tracker is built for this kind of engineering work. It delivers real-time position and orientation of tracked objects with extremely low latency, and keeps tracking even when markers are visible to just one camera. The Vicon DataStream SDK then streams that data into the robot’s control stack, with support for C++, Python, MATLAB, .NET, and LabVIEW. Tracker also streams to ROS and ROS 2, so mocap data can feed a teleoperation pipeline directly. 

Researchers at the Technical University of Munich (TUM) show what this looks like in practice. Their Geriatronics team is developing GARMI, a 1.6-meter humanoid for elderly care, using a 13-camera Vicon Vero system. By mapping Vicon markers on a data glove to GARMI’s 16 finger joints, the team can transfer human gestures to the robot’s hands. They are now working on controlling GARMI’s individual fingers in real time. 

“We need 3D high frequencies to track fast movements, and Vicon was super precise,” says Simone Nertinger, who leads user-perception research on the project. 

• Teleoperation is the remote control of a robot by a human, with feedback flowing back to the operator. 

• Teleoperation systems range from direct and shared control to VR and motion-capture-based setups. 

• Teleoperated robots already work in surgery, bomb disposal, deep-sea inspection, nuclear sites and space. 

• Humanoid robot teleoperation maps human body and hand movements onto a human-shaped robot through motion retargeting. 

• Every teleoperated task is training data. Demonstrations feed imitation learning, a core method in embodied AI. 

• Data quality decides robot quality. Precise, low-latency motion capture with Vicon Tracker and the DataStream SDK gives teams clean demonstrations and reliable real-time control. 

Building a teleoperation or robot learning setup? Explore Vicon for robotics.