Yahoo Finance Lip-Sync Technology Advances With ADR Improvements
As diligently as actors and crew work on set, it’s commonplace to re-record some lines down the road. For the teams behind “Eurovision Song Contest: The Story of Fire Saga” and “The Trial of the Chicago 7” the process of painstakingly matching audio and visual was further complicated by working remotely during COVID.
Eliza Pollack Zebert, supervising ADR and dialogue editor on “Eurovision,” says the L.A.-based team ran remote recording sessions. At times, they even had actors in entirely different countries, with lead Rachel McAdams in Atlanta and her singing stand-in Molly Sanden in Sweden.
More from Variety
Could 'Trial of the Chicago 7' Only Win Best Picture at the Oscars?
Oscars Predictions: Best Picture - SAGs Give 'Trial of the Chicago 7' an Oscar Boost
With SAG Awards Next, Does 'Nomadland' Have the Oscars Wrapped Up Without Cast Ensemble?
While all watching the same playback, the post-production team coached the pair through re-recording the lines and songs as exactingly as possible to match the syllables and intonations from McAdams’ original take.
Later, the team scrupulously plugged in the dialogue while editing fractions of a frame at a time, digitally stretching and compressing sounds for the lips to sync perfectly. While there is computerized tech that can theoretically match sound waves, it’s really no match for the human touch.
To help further construct the illusion of a single singer, the music editors bookended Sanden’s singing with McAdams’ inhales, exhales and early part of the take to blend the pair’s vocals.
“You have to [balance] what’s going to sound right and what’s going to look right at the same time,” says Zebert of the editing.
Oscar nominee Julian Slater, re-recording mixer on “Chicago 7,” worked under similar conditions with cast recording their lines from disparate areas such as home closets while using iPads for playback.
Then, the team went through the film second by second, deciding when to replace entire sentences versus single words, or at times even single syllables.
“We always do whatever we can to protect the original performance, [because] not only have [the actors] put a lot of heart and soul into it, but the director has then chosen that desired take.”
When it comes time to send blockbusters abroad, companies such as Pixelogic step in for the international versioning process. Andy Scade, Pixelogic’s senior VP and GM of worldwide digital cinema and home entertainment mastering services, has worked on a slew of films including Oscar-winners “Jojo Rabbit” and “The Favourite.”
Prepping projects for international exhibition may involve everything from translation and dubbing to creating versions differentiated by elements including sound, formatting and frame speed for each individual exhibitor. While it’s commonplace to have as many as 500- to 600-plus adaptations for each title, that was a sheer impossibility when film was used as a medium; the reels
were too expensive to manufacture and deliver worldwide.
Additional layers of customization mean “you can bring your content to an audience and have it speak to them in a way that is much more meaningful by virtue of localization of that material,” says Scade. Part of that process is ensuring the translations maintain the same integrity as the original with regard to humor and intent. Beyond that, matching up the audio and visual for differing languages is a laborious process.
While it may seem that computers can handle the heavy lifting on this front, it comes down to everyone working hard to match the original cast regardless of the source language.
As for the deep fakes that have overtaken the news recently, Scade speculates that they’re fabricated with paint-over visual effects technology. Though that could theoretically work for feature films, “the timing and the cost implications of doing it [are] not really permissible to enable this happening at scale yet,” he notes.
For now, putting words in someone’s mouth is remains a tricky proposition, no matter which language is spoken.
Best of Variety
Sign up for Variety’s Newsletter. For the latest news, follow us on Facebook, Twitter, and Instagram.
