One of the major challenges in the film industry and TV series today is the abundance of high-quality films and series that are not in the native language of the country in which they are shown, particularly in the English language, leading to viewer dissatisfaction with having to read subtitles. Hudson AI, a startup under Creative Labs by Samsung in South Korea, is attempting to address this issue using AI.
A survey by Morning Consult in 15 countries worldwide found that only viewers in South Korea and China reported high satisfaction with reading subtitles, up to 70%. Whereas in Western countries like the United States or Europe, viewers prefer dubbed audio more.
Hyunjin Shin, the CEO and founder of Hudson AI, stated that not every film or series, no matter how critically acclaimed, is guaranteed to be purchased for distribution in a specific country, along with the investment required for translation and dubbing. This was exemplified by the case of South Korea’s Parasite, an Academy Award winner, which had limited theatrical releases in the United States with only subtitles (Bong Joon-ho, the director, referred to subtitles as a mere “one-inch barrier” during his Oscar speech). This issue has become a barrier for international film distributors.
Hudson AI was established to address this challenge, with the original team working at Samsung Electronics in South Korea and presenting this idea before being selected to join the Creative Labs (C-Labs) accelerator program. The goal is to tackle the language barrier in the film industry using AI to translate dialogue and synchronize lip movements to the native language.
The voice conversion process involves three main components:
1. Translating language using existing LLM models in the market, along with allowing language experts to refine the final output.
2. Voice Conversion: Hudson AI develops a text-to-speech model that can learn and maintain the original actor’s tone, pitch, and vocal nuances to preserve the scene’s authenticity.
3. Lip Sync: Another AI model developed by Hudson AI adjusts the facial expressions of actors to match the translated dialogue accurately.
Shin highlights that the standout feature of Hudson AI is the naturalness of voice and lip movements of actors. While text-to-speech technology has been around for a while, it has not been successful in conveying emotion effectively, which is crucial for storytelling. He also emphasizes that the company’s goal is not to replace voice actors but to simply facilitate and enhance the viewing experience of foreign films and series.
Currently, Hudson AI has recently secured 700 million won (approximately 18 million USD) in seed funding and has begun testing with film companies domestically and internationally, with plans to offer services in a Software as a Service (SaaS) format.
TLDR: Hudson AI is using AI to tackle the language barrier in the film industry by translating dialogue and synchronizing lip movements to the native language, aiming to enhance the viewing experience of foreign films and series.
Leave a Comment