Amazon Prime Video is using AI lip-syncing for dubbed titles, starting with Maxton Hall

In a bizarre turn of events, Amazon Prime Video is using AI to make dubbed shows look more natural, starting with its hugely popular German series Maxton Hall.

The streaming service is using a combination of AI and VFX technology to change the way actors’ mouths move on screen so they match the dialogue in dubbed versions. Basically, instead of watching an actor speak German while hearing a completely different English voice, the new technology is designed to make their lip movements line up more closely with the English dialogue.

Variety reported that the feature is now available on Maxton Hall seasons 1 and 2 globally in English. It will also be used for the third and final season when it releases on Prime Video on 9 December.

Maxton Hall is a German-language romance series. As per Amazon, it is currently Prime Video’s most-watched international original series to date.

‘We’re making it easier for their stories to reach an even broader and global audience’

The company said the dubbed versions uses human-dubbed audio. But the AI technology then synchronises the actors’ mouths to that audio. The aim is to create “a seamless viewing experience” and allow viewers to stay more immersed in the story.

Raf Soltanovich, vice president of technology at Prime Video and Amazon MGM Studios, said, “At Prime Video, we’re constantly innovating with new tools that enhance the viewing experience for our customers.”

He added, “With lip-syncing, we’re creating a more seamless and immersive way for customers to enjoy global content.” 

Soltanovich continued, “And for creators, we’re making it easier for their stories to reach an even broader and global audience. All done under creative oversight to ensure the integrity of their artistic vision is preserved.”

Amazon has not revealed exactly which AI tools or systems were used to create the technology. For now, Maxton Hall is the first title to get the treatment. But it sounds like it won’t be the last.

Amazon Prime Video also plans to expand the technology to additional titles in the future. It said it is continuing to explore “practical and useful applications of AI that help enhance the viewing experience, always with the customer at the centre and creators in the driver’s seat.”

I don’t know who asked for this because I was perfectly happy watching it with subtitles anyway.

For all the latest film and TV updates and hot takes, like our Facebook page.