Preserve emotion and tone across languages
A good dub doesn't just translate the words — it carries the speaker's energy, emphasis, and mood into the new language. This article explains how Dubly.AI transfers emotion and what you can do to get the best result.
How emotion transfer actually works
When you use Original or Studio-Like as your voice cloning style, Dubly.AI clones the speaker's voice from your source audio. The clone captures more than just timbre — it captures:
Pitch and melody — how the voice rises and falls across a sentence.
Pace and rhythm — where the speaker slows down, speeds up, or pauses.
Emphasis — which syllables or words the speaker stresses.
Breathing and energy — subtle cues that make a voice sound confident, tired, excited, or calm. When the translation is synthesized, these characteristics are applied to the new language. The dubbed line ends up with a similar emotional contour to the original — even though the words and the rhythm of the target language are different. There are no manual emotion controls (no "happy", "sad", "excited" tags). Emotion transfer is automatic and driven entirely by your source audio plus the voice model.
What you can do to help
1. Give the model clean source audio
Emotion is in the nuance — and nuance is the first thing that gets lost to bad audio. Upload source material with:
A clean voice track (lavalier or shotgun microphone, quiet room).
Background music at least 12 dB below the speaker during dialogue.
No heavy voice effects (deep reverb, telephone filter, auto-tune). If you have to choose between shorter-but-clean and longer-but-noisy, go clean. The clone sounds more expressive on 60 seconds of pristine audio than 10 minutes of background noise.
2. Punctuate the source text well
When the AI re-translates and re-synthesizes a sentence, it uses punctuation to decide rhythm:
Commas create short breath breaks.
Periods create full stops and reset the melody.
Question marks lift the pitch at the end.
Exclamation marks drive the energy up. If you find yourself fixing a flat-sounding dubbed line in the transcript editor, check the source text's punctuation first. A missing comma at the right spot often revives a whole line.
3. Use the tone field in your Translation Style
A Translation Style has a Tone field (professional, casual, energetic, etc.) and a Formality setting. These shape the translator's word choice — which in turn shapes what the voice model has to say. An energetic tone with vivid verbs gives the voice more to work with than a flat, literal translation. See Maintaining Brand Language and Terminology Databases for how to set this up.
4. Pick the right cloning mode
Original preserves the full range of the speaker's emotion — including rough edges. Use when authenticity matters more than polish.
Studio-Like smooths the voice into a cleaner sound. You keep the speaker's voice identity but lose a bit of the raw emotional texture. Attention: Do not use if you already have clear audio quality
Replace Voice uses a library voice. Emotion comes from the library voice's natural delivery and the translated text's rhythm — the original speaker's emotion is not transferred, because their voice isn't used at all.
Where emotion transfer has limits
Even at its best, emotion transfer is a close approximation, not a perfect copy:
Whispered, shouted, or extreme voice states can soften in the clone. If a scene depends on a whispered confession or a full shout, expect some loss.
Very fast speech may feel slightly flatter in the target language — some languages need more syllables to say the same thing, and the model has to fit them into the original timing.
Emotional laughs, cries, and vocalizations are not synthesized. These stay in the source audio only; the translated dialogue around them is dubbed.
Multi-speaker overlap in the source can confuse the cloner — the resulting emotion may blend two voices. Record cleanly separated dialogue when you can.
Fine-tuning after the dub
If a specific sentence in the dub feels emotionally off:
Open the Edit Translation tab.
Find the sequence and play both the original and dubbed audio.
Adjust the target text — reword for punch, add a comma for breath, or split a run-on sentence into two. Save and let the audio re-synthesize. It's usually two or three sequences per dub that need a touch-up — not the whole video.
The short version
Dubly.AI transfers emotion automatically through voice cloning. Your job is to give it good raw material (clean audio, clear punctuation, a decent Translation Style) and fix the outliers in the transcript editor afterwards. You don't need to tag emotion manually — the voice does the work.