As more and more viewers watch international TV shows and movies via streaming, Israel-based company Adapt Entertainment has found a way for programming to literally speak to everyone.
According to company founder Darryl Marks, fans often complain about how badly dubbed shows and movies are. Adapt’s technology combines AI and visual effects to seamlessly translate movie dialogue into English and other languages.
Most recently, the technology was used in the English version of Maciej Barczewski’s film The Champion, about a prizefighter who must win matches to survive in Auschwitz.
Year. Barczewski calls the technology a “game changer.” Original films are in Polish and German. The director wanted to shoot in English, but there was no budget.
Mike Seymour, the film’s VFX supervisor and Adapt technical advisor, explains that the work and synchronization happens after picture lock.
Usually re-recordings are done by voice-over actors, but in this rare case Barczewski was able to bring back the cast and re-record the dialogue track in English. The tracks were then perfectly synchronized and processed to the original film using AI neural rendering technology based on the software system PLATO (Physics Learning with Automatic Object Encoding and Tracking).
Seymour describes the process as follows: The actors’ faces are then spoken in English throughout the film using advanced AI and machine learning. It’s all visually constructed from an actor recording dialogue in his sound studio. ”
The combination of AI and PLATO visual effects preserves the original performance subtext. “You have to be true to performance, not arbitrary technical specs,” says Seymour.
The production time for “The Champion” took a total of three months, including one week for the actors to shoot.
Re-recording the dialogue track dramatically reduced the cost per shot of large scale dialogue replacements.
Seymour says the company has no intention of replacing subtitles. “These are invaluable to the deaf community,” he says. Rather, Adapt “opens up the world of foreign cinema without taking the viewer away from the cinematic experience, and does so in a manner designed to respect the cast’s acting choices, the writer’s intent, and the creative.” Aiming for the director’s vision.”
