Meta's new AI model is a real-time translation expert
Meta's latest AI output is a major advancement for real-time text and speech translation.
On Tuesday, the company released SeamlessM4T: a multimodal model that translates text to speech and vice versa. Meta claims SeamlessM4T is "the first all-in-one multilingual multimodal AI translation and transcription model," meaning it is uniquely able to translate and transcribe languages at the same time. SeamlessM4T can translate speech-to-text, speech-to-speech, text-to-speech, and text-to-text inputs for up to 100 languages. Translations for speech-to-speech and text-to-speech translations outputs support 35 languages.
This Tweet is currently unavailable. It might be loading or has been removed.
Like other AI models recently released by Meta, including Llama 2 and AudioCraft, SeamlessM4T is publicly available for researchers and developers with a research license. Alongside the model, Meta is also releasing its training dataset called SeamlessAlign, which has 270,000 hours of speech and text alignments. Unlike OpenAI and Google, Meta has made a point of making its models open-source and publicly available. Meta's approach of launching open-source models has the dual effect of enabling developers to build and improve the products, while also winning points amongst AI ethicists who are calling for transparency of generative AI systems.
You May Also Like
Meta's open-source approach may seem altruistic, but it's a strategic power move in a ruthlessly competitive market against other big tech companies developing AI. There's also the issue of data collection that all AI models must contend with. According to the blog post, SeamlessM4T's dataset (SeamlessAlign) consists of publicly available data, there are ethical and legal issues surrounding use of copyrighted works and personal data without consent.
Meta's announcement didn't detail specific plans for SeamlessM4T, only hinting that it wants "to explore how this foundational model can enable new communication capabilities." In other words, we might someday see a consumer-facing version of SeamlessM4T on WhatsApp or Instagram.
Topics Artificial Intelligence Meta
Cecily is a tech reporter at Mashable who covers AI, Apple, and emerging tech trends. Before getting her master's degree at Columbia Journalism School, she spent several years working with startups and social impact businesses for Unreasonable Group and B Lab. Before that, she co-founded a startup consulting business for emerging entrepreneurial hubs in South America, Europe, and Asia. You can find her on X at @cecily_mauran.
