KARA IS CREATING SIGN LANGUAGE AVATARS THAT COULD SAVE LIVES
The coronavirus pandemic has shown how important it is for governments to be able to communicate quickly and clearly in an emergency. Tech startup Kara is using motion capture to make that communication faster and more accessible for the Deaf community.
“Kara started with a core vision to improve the access of the Deaf community to the incredibly broad range of information, content and services around the world,” says Arash Tayebi, Co-Founder and CEO of Kara.
That vision was born after Tayebi was diagnosed with Menieres, a condition that caused him partial hearing loss, and he became aware of the access barriers faced by deaf people in education and across other walks of life.
“Sign language is the primary language of the Deaf people, and it is a moral and sometimes legal obligation to provide access to signed content,” Tayebi says. “Our team’s focus is to change the perspective of how avatars are used in movies, dancing, sports, gaming, and advertisements.
“We use the world’s leading software and hardware technologies, combined with our in-house capabilities, to deliver high-demand, broadcast-quality translations in any signed language using digital avatars.”
Kara operates Vicon Vero cameras with Shōgun, using the data to animate avatars created with Epic’s MetaHuman technology. “The input to our system is sign language gloss text. The output consists of individual motion-captured animations of each word that are smoothly blended together using Kara’s proprietary algorithms,” Tayebi explains.
SUBTLETIES OF EXPRESSION
It’s important that sign language avatars are not only functional, but expressive and fluid. Failing to capture the nuances of human sign language interpreters in Kara’s avatars would result in animations equivalent to the flat delivery of early text-to-speech software for those with hearing.
“To be able to provide our services effectively, the most viable solution is using motion capture instead of hand animation,” says Tayebi. “Most importantly, to capture the nuances of sign languages, motion capture is of the utmost importance. It’s essential that it’s reliable and extremely accurate.
“However, quite early on in our work, we realized the limitations of many off-the-shelf systems around the capture of nuanced body- and finger-movements. Vicon is the best motion capture technology for our requirements. It helps us capture the smallest motions of a signer with minimal animation cleanup needed.”
The advantage of using motion capture isn’t only the fidelity it offers. “Vicon provides us with the ability to produce content at a much faster rate and much more cost-effectively,” says Tayebi. “With Vicon Vero cameras and Shōgun software, our process of animating our digital avatars is much more automatic.
“We now rely much more on motion capture, whether for long pieces of content or individual words, and much less on artistic interpretation to drive our avatars. This means that our avatars will be much more true-to-life and the sign language content produced will be of a higher quality. We can most definitely stand by the statement that these are hyper-realistic digital avatars/ humans.”
PRACTICAL APPLICATIONS
At the moment Kara has two main applications for its technology: producing signed versions of children’s books such as a New Zealand Sign Language (NZSL) translated version of The Very Hungry Caterpillar, and sign language emergency notification messaging systems in the US and New Zealand.
“In an emergency situation, whether it is a natural disaster such as a tsunami, or a police alert, it’s very important to send the messages out to all receivers in a timely manner,” explains Tayebi. “Most emergency notification systems currently heavily focus on audio alerts. In some instances they involve textbased alert systems, which incorrectly assume a certain level of text-based comprehension by the Deaf.
“In the case of emergency situations, speed is of the utmost importance. We have worked alongside the Deaf community and have developed a system where a sign language avatar is able to deliver an emergency text alert accurately in sign language in a matter of minutes, without having to rely on the delayed availability of interpreters in the crucial moments after a disaster occurs.
“For scenarios such as emergency alerts, a message in a signed language can be sent out in a matter of minutes.” Kara is currently in talks with a number of governments at both local and national levels, working with them to integrate their technology into preexisting emergency alert systems. Soon, it could be saving lives.
While Kara needs government buyin, however, the company’s most meaningful conversations are with its most important stakeholders: the Deaf community.
“Sign language belongs to the Deaf community and they have the right to create and work with the technology to ensure accurate translations,” says Tayebi. “The best way to do something for a group of people is to have them be involved at every point in the process, and the Deaf community is no exception to that. Co-creation is an important part of creating something that will make an impact.”
