Through Microsoft Azure and its AI capabilities, Peloton develops live captions for members who are deaf or hard of hearing

For David Wolfe Rose, that day in early February was like all Christmas at once, like having a new kitten, like all special days rolled into one. A través de Microsoft Azure y sus capacidades de inteligencia artificial, Peloton desarrolla subtítulos en vivo para miembros sordos o con problemas de audición A través de Microsoft Azure y sus capacidades de inteligencia artificial, Peloton desarrolla subtítulos en vivo para miembros sordos o con problemas de audición

Rose had prepared a room in the back of his house in preparation for the delivery. That morning, she looked forward to it. Two men soon arrived, assembled Rose's new indoor stationary Peloton bike, and showed her how to use it. Rose was, well, beyond elated.

"It was like getting married again," she says. “If I'm honest, it was like having another baby. It was very exciting when the bike arrived.”

Peloton members are known for their dedication to the brand's energetic online classes. But for Rose, the high-tech bike was more than just challenging exercise. She promised connection and community. It was a gateway to a place where she could belong.

Rose, who is deaf, bought the bike after learning Peloton offered closed captioning for its classes on demand. Most recently, using Microsoft's Azure Cognitive Services speech-to-text and translation technology, the company launched live captions in September for its live classes to improve accessibility for members who are deaf or hard of hearing. hearing.

Suresh Bathini, vice president of software engineering at Peloton, says the New York City-based company decided to work with Microsoft to expand accessibility to its platform. After hearing a member sign up for full live classes for his partner, who is deaf, he knew doing so was the right move for the Peloton community, he says.

“That led us to provide closed captioning for people taking live classes. We want to provide immersive experiences that are accessible to all users,” says Bathini.

Previously, the company had provided closed captioning only for its on-demand classes. But that meant the live experience so beloved by members was not available to those who are deaf or hard of hearing.

While the decision to introduce live subtitles was clear, the execution of that vision was a bit more murky. A primary challenge was determining how automated speech recognition software could facilitate Peloton-specific vocabulary, including number phrases used for class countdowns and for setting resistance and cadence levels. Latency was another issue: subtitles wouldn't be very useful, after all, if they lagged behind what the instructors were saying.

Bathini says that Peloton chose Azure Cognitive Services because it was cost-effective and allowed him to customize his own machine learning model to convert speech to text, and it was much faster than other solutions on the market. Microsoft also provided a team of engineers who worked alongside Peloton throughout the development process.

“Having a support system, especially on the engineering and software side, helped us speed up the solution,” says Bathini. “It was a very collaborative process.”

A través de Microsoft Azure y sus capacidades de inteligencia artificial, Peloton desarrolla subtítulos en vivo para miembros sordos o con problemas de audición

Eric Boyd, corporate vice president of Microsoft Azure AI, says that Peloton's use of Azure Cognitive Services is a great example of using AI to break down barriers and address inequalities.

“It's great to see Peloton embrace the power of AI to make their platform more accessible to everyone,” he says. “Through Cognitive Service for Speech, part of the Azure AI platform, Peloton was able to develop and implement live captions for its classes, creating a more accessible and engaging experience for its community of members, especially those who are deaf or hard of hearing. hearing problems.

"This collaboration perfectly embodies our mission to empower every person and organization on the planet to achieve more."

For Rose, those captions on her bike screen have been nothing short of transformative. She had joined various gyms over the past 20 years, worked with personal trainers and lip-read her instructions and guidance. But she couldn't interact much with other gym members, other than the occasional smile or thumbs up. After a while, feeling isolated and left out, Rose's motivation invariably fizzled out and he stopped going.

"I've always felt a bit left out in gyms, but with Peloton, it makes me feel like I'm 100% part of the community because I can participate and follow what's going on," says Rose, who lives in Telford, a city in central England.

His lack of motivation has evaporated, along with about 20 pounds. Since February, Rose has racked up more than 900 (and counting) rides, joined a Peloton Facebook group, and regularly chats online with other Peloton members on topics ranging from team issues to loss goals. weight.

“It's a small community, which is nice,” he says.

When Rose first got the bike, he tried the live classes because he wanted the immediacy and feeling of connection. He could look at the leaderboard showing the results of the participants and see where he ranked, give another member an award, and most importantly, keep an eye out for the award he was hoping to receive from the instructor.

Later, Rose would go back and redo the class with closed captioning just to see if he got an award. At the beginning of October, he still expected and wanted it. But without subtitles, Rose could only watch the instructors and try to copy what they were doing. The introduction of live subtitles, he says, "blew me away completely."

“Now I have the opportunity to do the class live and wait for that recognition, instead of having to go back and watch it a second time,” says Rose, 60, a senior lecturer in deaf studies and British Sign Language. / English interpreting at the University of Wolverhampton.

Meryl Evans previously worked in a gym for about four years, but like Rose, she was ready for a change and wanted more training guidance from her. A trainer in the studio would give Evans, who is deaf, a printed sheet of instructions to follow. But they were basic and lacked the kind of detail she wanted: keeping your back straight, tightening your core, and that maximizes workouts and can help prevent injuries.

When the pandemic hit and gyms closed, Evans decided to find a subtitled workout she could do at home. Workouts without subtitles weren't an option, although Evans' eyes are her “number one listening tool,” she finds it difficult to read lips on video, let alone while she's concentrating on a workout.

Evans tried out Peloton and a competitor company and liked the variety of Peloton's classes, ranging from cycling to boot camp to yoga, and the diversity of its instructors. But it was Peloton's subtitles, which were more visible and easier to read than the other company's, that sealed the deal.

“My workouts are so much more efficient now because I have those instructions in subtitles,” says Evans, 51, a digital marketer and accessibility consultant who lives in Plano, Texas.

“And it has made a big difference. My back doesn't hurt as much as it did because of those instructions."

A lifelong runner and sports enthusiast, Evans started out on a treadmill, but soon switched to a Peloton Bike that she obtained through the company's Comeback Program, which recognizes members who have overcome adversity. .

Like many people, both Evans and Rose like to work out to music. Evans is a fan of Broadway, especially "Hamilton," and discovers new music primarily through closed captioning in Peloton classes or on television shows. Rose leans towards 80's rock tunes, she loves the group Queen and likes to turn her wireless speaker up loud enough to feel the beat. Knowing what song is playing during workouts provides an added boost, they both say.

"Even though I can't hear it, if I know a song, my head will play it," says Evans. "I hear it in my head while I'm driving."

Evans and Rose hope Peloton will add song titles to its live captions, something Bathini says the company is looking to do, and Evans would also like to see the addition of one or two instructors with disabilities. Until now, live captioning for Peloton classes is only available in English, but the goal is to make Peloton classes as accessible as possible, says Bathini.

“We want fitness to be accessible to all of our members. Our mission is truly to power the future of fitness for the world through connected experiences,” she says.

Both Evans and Rose praise Peloton for taking steps to meet the needs of people with disabilities and say other companies, in the fitness industry and beyond, should do the same. Closed captioning is used even by people who aren't deaf or hard of hearing, notes Evans (Ever watch a football game in a noisy pub?), as are devices originally designed for people with disabilities, such as the elevators.

Peloton, which offers closed captioning, "makes a difference, because it shows that the company is serious about accessibility," says Evans. “The fitness industry has a long way to go, but Peloton is perhaps ahead of the pack because of what they do with subtitles.

"I'm very loyal to Peloton," she says. "I have no desire to change."

And neither did Rose, who ultimately found her passion for training through a high-tech Internet-connected bike and a community of like-minded people. Even if other fitness companies follow in Peloton's footsteps, he plans to stick with them out of "respect and loyalty."

And he has another reason to stay engaged. One day in late October, Rose jumped on her bike for a 30-minute ride to songs from the 2010s. As class progressed, a legend appeared. It was the long-awaited recognition that he was congratulating Rose on her 900th trip.

In that small room, on the bike that had brought her connection and a newfound passion, Rose squealed with joy.

She has already started thinking about his thousandth trip.

Featured photo: David Wolfe Rose works out on his Peloton Bike + at his home in Telford, England. (Courtesy of David Wolfe Rose)