People with hearing loss deserve to enjoy films and theatre too

By Richard Pallardy

Built For Good, a United Kingdom-based technology company, aims to enhance the enjoyment of films and live performances for people who experience hearing loss by offering real-time closed captioning on smart glasses. The system for theatre is called WatchWord Live. It tracks the audio from a stage and projects pre-prepared text of the dialogue onto the lens, allowing those who have difficulty hearing to place the captions where they want and follow along without interruption.

Challenges for the hard of hearing and deaf

“People who need captions cannot attend everything they want to see, because captions are not available when they want them to be.”

Tabitha Allum, co-founder and Chief Operating Officer of Built For Good

Accessing films and live performances has been a perennial challenge for those who experience hearing loss, a common problem for older people in particular. Accommodations for this invisible disability have included light-emitting diode (LED) displays, which show the text of the performance.

However, LED screens may be difficult to view, depending on their position in the theatre or auditorium, and are typically only available at specific times, like a weekday matinee performance, which may be inconvenient.

Where dinner and a film or theatre show is a typical date night, a weekday lunch and a matinee viewing are not. Society should not only offer cultural experiences to people who are able to listen to the audio of a film or performance without assistance.

This is a further problem for one-off events, such as lectures or debates, which are less predictable. It is harder to offer closed captioning for live conversations. Even for scripted events, there are mechanical challenges.

“Often, you have to turn your head to read the captions and then look back at the stage, which means you miss 50% of the action,” says Tabitha Allum, co-founder and Chief Operating Officer of Built For Good, a company that offers a solution for those with hearing loss.

Fixing accessibility issues

“The system is listening to what is happening on the stage, performance after performance. It can track the show.”

Allum

Allum and her partners have been working on practical solutions for the hard of hearing and deaf for over a decade. They landed on the use of a pair of glasses to offer line-of-sight accessibility, resulting in a seamless experience to the viewer regardless of where they choose to sit.

The specs are designed by Epson, a company best known for its printers. They easily slide over vision correcting glasses, which is essential to the ageing population, many of whom rely on prescription glasses to see.

By reading the text on their lenses, the viewer can follow the film or the play. Text is usually available as part of the digital playback system for films, whilst theatres can upload the text of a play with estimate timings and example audio to create a repeatable file. Sound cues, such as offstage gunshots or other noises, can be included so that the audience member is aware of all auditory inputs. For live, unscripted events, stenographers may be brought in to generate the text.

Users can personalise the output using the handheld controller, similar in design to a smartphone. It allows them to adjust the size and colour of the text according to their preference using a simple interface.

“You end up with a really crisp display. You can make the text as small or as large as you want. You do not lose resolution,” Allum explains. “You can control the depth of field, meaning where the captions lie comfortably in space between where you are sitting and what you are watching.”

Challenges to implementation

“We would love to be able to invest in the creation of a bespoke pair of glasses for use in cultural settings, but we are able to use an existing off-the-shelf product to provide a really good service. People are very happy with it.”

Allum

Theatre performances require a bit more work on the front end. However, the text output is adaptable. Once the initial text is input, the software can time its display according to the actual speech onstage for a play. Vocal frequencies in the higher range may be challenging for those with hearing loss to pick up, but being able to read the text as you listen to the actors typically solves this issue.

“There might be some performances where there is more laughter in the audience, so the actors have to pause longer,” Allum says. “The captions respond to what is actually happening in the room.”

Built For Good seems to have solved one of the major mechanical challenges by using the Epson glasses. Fitting them over existing prescription glasses has been crucial and the models they have chosen do so without problems for most users.

Their collaboration with Eureka’s Innowwide programme, which provides grants to SMEs and startups to explore new regions, has allowed them to expand into the United States.

“We have been able to cooperate with a couple of organisations who can work on our behalf, more locally, which will save us getting on a plane every two minutes to demonstrate our product,” Allum says. These organisations market the smart glasses to theatres.

She notes that theatres in the United States show their dedication to ensuring that audience members have a positive experience through their staffing structures, which often include senior roles like a Director of Patron Services. This set up has further facilitated Built For Good’s entry into the market. The company has been happy with their Eureka project.

“The level of monitoring on Eureka’s side is really light-touch, which is brilliant,” she observes, citing Built For Good’s small staff and limited time for communication.

Users enthusing

“While there is a moral case for providing better accessibility, there is also an economic case for bringing people who experience hearing loss back to the theatre.”

Allum

Those who have tried the glasses have provided positive feedback to Built For Good. Their experiences of both film and live performances have been improved by using the technology. Many of them have partial hearing loss and are thus able to track some of the sound cues. The text enables them to complete the mental circuit; a faintly heard word seamlessly registers on the screen and viewers do not have to struggle to compute what it actually meant.

The adaptability of the readouts on the lenses has been a particularly appealing aspect for many.

“One person will use pink, enormous text. Another person will use green text that is tiny and in the corner of the lens. It all depends on their eyesight and how reliant they are on reading every single word,” Allum explains.

In the case of theatre especially, many users are life-long enthusiasts who felt unable to enjoy one of their passions due to their hearing loss. It was simply not worth the hassle and expense when they could not hear the voices of the actors. A pair of glasses with a bit of tech embedded allowed them to regain that pleasure.

Allum describes the enthusiasm from people who do not have hearing disabilities as well. They frequently approach booths the company sets up at theatre performances and indicate that they will tell friends and relatives about the option.

Future applications

“We will be providing multilingual captions where people can select the language.”

Allum

Built For Good has anticipated additional uses for its technology. Non-English language speakers can also benefit from the text-based readouts on the lenses. Viewing the text can reinforce the user’s understanding of the dialogue or translate it for those who do not understand it at all.

“Even though this was made primarily as a device for deaf and hard of hearing people, I think the opportunity to bring audiences in who do not have English as their first language is huge,” Allum says.

The work underscores both the complexity and simplicity of accessibility issues. Once these problems are identified, it takes serious work to solve them. But once that work is done, the solutions themselves can be elegantly simple.