Researchers at Columbia Engineering have trained a human-like robot named Emo to lip-sync speech and songs by studying online videos, showing how machines can now learn complex human behaviour simply by observing it.
Emo is not a full humanoid body but a highly realistic robotic face built to explore how humans communicate. The face is covered with silicone skin and driven by 26 independently controlled facial motors that move the lips, jaw, and cheeks.
These motors allow Emo to form detailed mouth shapes that cover 24 consonants and 16 vowels, which is critical for natural speech and singing. The goal was to reduce the uncanny valley effect, where robots look almost human but still feel unsettling because their facial movements do not match their voice.
How Emo learned to lip sync like a human
The learning process happened in stages. First, Emo explored its own face by moving its motors while watching itself in a mirror. This helped the system understand how motor commands change facial shapes.
Researchers then introduced a learning pipeline that connects sound to movement. Emo watched hours of YouTube videos of people speaking and singing, while an AI model analysed the relationship between audio and visible lip motion.
Instead of focusing on language or meaning, the system studied the raw sounds of speech. A facial action transformer converted those learned patterns into real-time motor commands.
This approach allowed Emo to lip sync not only in English but also in languages it was never trained on, including French, Arabic, and Chinese. The same method worked for singing, which is harder because of stretched vowels and rhythm changes.
Researchers say this matters because future robots will need to communicate naturally if they are going to work alongside people. This advancement has arrived when interest in robots for homes and workplaces is climbing fast.
At CES 2026, that momentum was on full display, with demos ranging from Boston Dynamics’ Atlas humanoid which is ready to enter workplace to SwitchBot’s household-focused robot that can cook meals and do your laundry, and LG’s upcoming home assistant robot designed to make everyday life easier.
Add advances like artificial skin that gives robots human-like sensitivity, and paired with realistic lip syncing, it is easy to see how robots are starting to feel less like machines and more like social companions. Emo is still a research project, but it shows how robots may one day learn human skills the same way we do by watching and listening.
