Vertical dance study shows link between perception and action experience
In the videos produced for The Vertical Dance Project a series of dots shift hypnotically, conjuring something very recognizable from incredibly little visual information: human movement. The project, a collaboration between researchers at the universities of Toronto, Wyoming and Shanghai, has used a Vicon system to yield powerful insights into human perception that could have widespread impact on the creation of digital humans.
“To talk about the significance of this project we need to first talk about the work by Gunnar Johansson in the 1970s,” explains Dr. Xiaoye Michael Wang, a postdoctoral fellow from the University of Toronto’s Faculty of Kinesiology. “He attached highly reflective markers onto people’s major joints, and then used high intensity spotlights so that he only captured the markers, and then he used the data to show that people can actually see meaning in movements, just based on the motion of a few dots. If they were static, people wouldn’t be able to see anything. Once they start moving, people can see it.”
Since Johansson’s work in the 70s, other researchers have discovered that inverting biological motion is very disruptive to people’s ability to understand these movements. With The Vertical Dance Project, Wang and his colleagues have probed the phenomenon deeper, seeking to better understand the mechanism behind it.
The specific format of the study was the result of serendipity. Dr. Qin “Arthur” Zhu, Division of Kinesiology and Health, University of Wyoming, attended a performance staged by his university’s Department of Theater and Dance. The production involved dancers tethered to a cliff face performing movements upside down and horizontally, using the vertical cliff face as if it were a floor.
“When I talked to the dancers,” says Zhu, “I asked if they had a hard time interpreting the inverted motion of their dancing partners, and they said no, we can read and anticipate their movements well. That sparked the idea. Maybe because they have the experience of inverted movement through vertical dance, their perception is unique and allows them to overcome the inversion effect.”
Zhu and Wang collaborated with Dr. Margaret Wilson, a professor from the university’s Department of Theater and Dance, to capture 10 pairs of dance movements that are spatially congruent or incongruent as point-light displays using eight Vicon Bonita cameras and Nexus software. Each movement was performed both on the ground and in the air, with the display of half the movements inverted to test the perception of typical dancers, vertical dancers and non-dancers on whether the movements had been inverted.
A powerful legacy
The team were pleased with the execution of the study. “What’s great about the Vicon system is that the accuracy of the data and the sampling frequency is really good, even with the legacy camera,” says Wang.
“Another way to see how accurate the Vicon system is, is that using the data we are able to replicate all the movements smoothly and accurately enough that dancers all understand that this is a human movement,” adds Zhu.
Indeed, the only notable challenge in the data collection came from just how well the cameras performed.
“What was challenging in the original data collection was the fact that the body hanging on the rope has a vibration or an oscillation,” says Wilson. “We finally came up with a tether to try to hold them still. But in spite of that, as you look at the data, it’s sensitive enough that you can see that gentle vibration in the movement if you understand that that’s a possibility.
“It’s so subtle that if you don’t think it’s a possibility, it’s not there. But I often wonder if that was a cue for the vertical dancers because they know there’s no way to stop your body. So what’s interesting about the motion recorded by the Vicon system is that you can see the oscillation in spite of our best efforts to try to limit it.”
The study found that vertical dancers were much better at recognizing the inverted movements than the other subjects, suggesting that their unique experience with observing and performing inverted movements does, indeed, enable observers to use dynamic information to identify artificial inversion of motion.
“What is consistent is the direction of gravity,” Wang explains. “We take gravity for granted, but if you perform a movement upright versus upside down, the direction of gravity can actually specify how we are supposed to move. That is the key, because vertical dancers have more experience in seeing and performing these movements, and therefore they’re probably more familiar with the interaction between the dance movements and the direction of gravity.”
The team has plans for a further study that would use a more comprehensive marker set to derive a subject’s center of mass. They would then perturb it to see at which point vertical dance perception is broken.
“We could use this to further elucidate the mechanism behind biological motion perception. We also plan to integrate those with eye-tracking, to see where people are looking and what kind of information they use,” Wang says.
Breaking out of academia
Wilson is excited by the implications of the study for her practice as a dance teacher. “What’s so intriguing is understanding what information we use to make sense of our world, or of movement. When we see a vertical dance performance, we see the whole body and it’s an unfamiliar experience to people who’ve never hung from a rope. When people have vertical dance experience, they have experienced the pressure in their heads and the pressure on their hips, so they can appreciate the movement.
“But I think more than that, as a teacher I’m considering, ‘what information am I using to make a judgment on aesthetics? What information am I using to look at injury risk? What information am I using to try to help this person master the skill in a more efficient way?”
Zhu sees applications for this research both in situations in which people have to work upside down, such as on construction sites, and in the tech world. While the life sciences and the world of video games, virtual reality and the metaverse might appear to be poles apart, he sees this research as something that could help developers bridge the uncanny valley. “When you see someone operating an avatar you can actually really tell whether this is animated by a person or animated by a computer in terms of the movement dynamics,” he says.
“And therefore I think what we’re doing here is we’re actually setting up the foundations in terms of what makes it authentic purely based on the movement. We’re not talking about how the avatar looks, because it can be a lion or an elf, but in terms of how it moves, and how, based on this piece of information, we can say this is a person, or this is not a person. I think what we’re doing here can offer a bit of insight into this area, and that can hopefully help people who are doing advanced technological R&D to incorporate that into their design of avatars.”
While the Vertical Dance Project has a very focused field of study, its applications are potentially broad. As Wilson puts it: “We need to be able to look across the room and see the human coming at us, but then also the tiger. We use very little information to read that. Even with the light point display, there are things that we as humans pick out, in terms of shoulder width or hip width or our physical stance, and so I think these experiments are giving us an understanding of how we filter a lot of information.”
For more on The Vertical Dance Project, see the team’s Nature article.
