University of Southampton researchers are creating a catalog of movements to help study the art of conducting

It begins with a single motion. A quick flick of a baton and the silence is broken. Another movement and the tempo picks up, then another and new sounds join. It is an auditory explosion generated by dozens of individuals acting together, all working to create something that can inspire the imagination. And at the head of it all is a single figure exerting their will on the music through a series of movements, demanding the attention of both the musicians and the audience alike.  

The act of conducting an orchestra is an art, with each practitioner finding their own approach and developing their own idiosyncrasies. Some exert a calming influence, accentuating the tempo with a delicate touch, while others attack with a gusto that emphasizes the power of the instruments. The best conductors are in demand around the world, and often receive top billing and salaries to match. Countless musicians aspire to wield the baton, yet many are left disappointed by the lack of resources available to help them achieve their goals. A new study on the nature of conducting hopes to gain a greater understanding of the artform, potentially leading to new ways for conductors to train.

Located in southern England, the University of Southampton is working on a project known as “Capturing the Contemporary Conductor,” led by Drs. Richard Polfreman, Cheryl Metcalf and Ben Oliver. Using high-speed motion capture cameras, the study aims to record and collect the movements of conductors down to the slightest twitch.

With funding courtesy of the British Academy and the Leverhulme Trust, the Southampton team is working to record precise 3D representations of the gestures used in conducting, from the hands, whole body and even facial expressions. When that data has been preserved and properly catalogued, researchers and aspiring conductors will be able to access it in order to gain greater insights into the biomechanical intricacies involved in the undertaking the artform.

There is no single “right” or “wrong” way to conduct, but there are specific gestures on which conductors rely that newcomers can study. A right-hander will traditionally hold a baton in that hand, which they use to control the tempo of the music, including signaling the beginning of an up or down-beat. Meanwhile, the left hand is free to take on other tasks. A conductor may point to individuals to give them their cue, or to tell sections when to taper off. It is a language of its own that transcends spoken words.

Initially, the Southampton team planned to create an exhaustive taxonomy of conducting gestures using traditional methods, but soon realized that the fluid nature of a conductor’s movements would make such an undertaking unreasonable. The smarter solution was to catalog the movements of multiple conductors using motion capture, something that had never been done before. This required cameras designed to capture the most precise movements of an individual from multiple angles. For that, the team looked to Vicon.

“In order to capture the movements, gestures and skills involved in conducting, we needed to capture from both high and low positions, including the body, face and both hands,” said Dr. Metcalf. “Our normal setup in the biomechanics lab at Southampton is great for many of our shoulder biomechanics work or gait analysis, but this project was unique and required something very specific. Vicon kindly brought extra cameras and helped us create a bespoke camera setup based on our capture needs.”

The Southampton research team has a proven history when it comes to using Vicon’s mocap cameras to capture data. During her Ph.D. dissertation in 2003, Dr. Metcalf utilized small volume motion capture to investigate the relationship between movement and function of the wrist and hand. She later applied those findings to a 2012 project titled “PianoHAWK,” in which passive markers were affixed to the hands of piano players to gather data on how pianists develop their skills and how that might help to improve methods for rehabilitating repetitive strain injuries (or help them alter their techniques to preclude injuries altogether).

Southampton has used Vicon technology beyond music as well. In 2016, a study titled “Life at the Cutting Edge” sought to develop methods for assessing skill and expertise within an archaeological context. Participants wore passive markers on their hands and wrists while peeling potatoes using replica prehistoric tools. The use-wear profiles of replica tools were then assessed to help understand the skills used during prehistoric times.

In all cases, the use of mocap has seen new technology re-examine and improve old practices.

To ensure that the “Capturing the Contemporary Conductor” study received sufficient data, conductors and musicians performed in Southampton’s research lab, which is outfitted with several Vicon T-Series cameras, one 360-degree video camera, stereo and close microphones, and a Microsoft motion-sensing Kinect 2 camera. Once the lab was set up to receive the musicians, the team recruited three subjects: Geoffrey Paterson, Holly Mathieson and Dr. Oliver (who pulled double duty as conductor and researcher).

Each conductor wore wireless EMG (electromyography) devices on their biceps and triceps in addition to markers on their faces, hands and body. The conductors then stood on force plates to measure the impact of each gesture. The research team next moved on to recording the facial gestures of each conductor. To accurately reflect this, a few different solutions were attempted with the assistance of Vicon. After trying different pieces of facial capture hardware, the team eventually settled on passive markers on the face of each subject.

For their mocap performances, Dr. Oliver composed a new three-movement piece titled “Captured: Three Mocap Experiments for Conductor and Small Ensemble,” scored for seven instruments: oboe, bass clarinet, trumpet, electric piano, percussion, violin and double bass. Each of the three movements is designed to challenge the conductors in different ways. For example, the first movement involves managing strange time signatures at different speeds, while the second movement requires the conductor to coordinate the entrances of the ensemble, but not necessarily to conduct them in a traditional sense.

In effect, the variety of musical materials across the three movements ensures that the conductors are required to perform a wide range of gestures to successfully coordinate the ensemble. Microphones were fixed above the conductors and near each instrument to capture detailed sound recordings via post-production software Steinberg Nuendo. The audio was then synchronized to the Vicon gear using SMPTE time code.

Partnering with Vicon allowed us to get the results we needed. Other mocap solutions either lack precision or require bulky sensors attached to subjects, which wasn’t feasible given the movements involved,” said Dr. Metcalf. “Vicon’s optical motion capture is currently the only solution that offers good precision and reliability, and minimizes interference with the movements.”

Southampton’s “Capturing the Contemporary Conductor” combines biomechanics and music using the most cutting-edge motion capture and motion data technology, alongside audio and video recordings. The results will offer researchers, aspiring conductors and fans of orchestral music a new resource, and it will be available through open online access.

The art of conducting may be hundreds of years old, but technology is helping bring it to a new generation for the first time.