01 · Baseline
LTX 2.3 Base
Text-conditioned baseline.
Supplementary Material
Generated examples with synchronized visual direction, dialogue timing, and ambient sound. Use headphones for the full audiovisual result.
Qualitative Results · Baseline Comparison
Selected 15-second prompts shown under matched screenplay conditioning. Both examples compare four generators. MiniMax-H3 scores and single-GPU generation times were measured with 14.375-second clips.
01 · Baseline
Text-conditioned baseline.
02 · Baseline
I2VA, conditioned on the first frame of the 14.375-second MiniMax-H3 clip.
03 · Baseline
MiniMax-H3 generation using its native prompt format. Lip Sync Score: 7.141.
04 · Our Method
Our joint audio-video generation. Lip Sync Score: 3.366.
Additional qualitative comparison
Character 1 continues speaking while the camera alternates between the speaker and Character 2’s reaction.
01 · Baseline
Text-conditioned joint audio-video baseline.
02 · Baseline
I2VA, conditioned on the first frame of the 14.375-second MiniMax-H3 clip.
03 · Baseline
Joint audio-video generation from the shared 15s screenplay. Lip Sync Score: 7.343.
04 · Our Method
Our joint audio-video generation. Lip Sync Score: 3.983.
Qualitative 15s Results
15 seconds
Two people sit opposite each other on a moving train.
Close-up of Character 1 looking down, then raising his eyes to look forward.
Close-up of Character 2 smiling and looking forward.
Close-up of Character 1 looking forward with a shifting expression.
Profile view of both characters facing each other, with a blurry green landscape passing by through the train window.
Character 2You’re going to move to India and become a guru. I’ll find myself in the mountains.
No speech
Character 1Well, maybe you’ll have a cup of coffee with me before you go.
Character 2Yeah, I’d like to do that and put it to the test.
Persistent rhythmic rattling and low rumbling of a moving train carriage.
15 seconds
Two men in fedoras and trench coats stand and interact on the deck of a moving ferry under a cloudy sky.
Medium shot of both men standing by a mooring bollard on the ship’s deck, with open ocean and a dark, cloudy sky behind them.
Medium close-up of the two men, framed against a chain-link fence and the white exterior of the ship’s cabin.
Wider medium shot of the men against open water and sky.
Medium close-up against the fence and cabin exterior.
Character 1It’s exactly square with Teddy Daniels, the man, the legend, I’ll give you that.
Character 2What the hell you boys smoking over there in Portland anyway?
Character 1Seattle. I came from the office in Seattle.
No speech
Character 1How long you been with the Marshals?
Persistent sound of rushing wind and the low rumble of a ship engine moving through water.
15 seconds
A tense, intimate interaction indoors featuring close-up shots of two injured individuals in a dimly lit setting.
Close-up of Character 1 resting her head, crying, and pressing a white tissue to her bleeding nose.
Close-up of Character 2 leaning in and looking downwards.
Return to Character 1 holding the tissue.
Close-up of Character 2 looking down.
Character 2 softens into a faint smile.
Character 1Why?
Character 2I’ll go see him.
Character 1And?
Character 2I’ll give him some bruises of his own and tell him to look for a guy called Teddy.
Character 1Oh, he’ll kill you, Lenny.
No speech
Character 2My wife used to call me Lenny. Yeah, I hated it.
Quiet indoor room tone.
15 seconds
Two men interact in a brightly lit stone hall with dark pillars.
Close-up of Character 1 looking slightly downward while reacting.
Close-up of Character 2 looking upwards off-screen and reacting.
Close-up of Character 1 looking forward and reacting.
High-angle over-the-shoulder shot from behind Character 1, looking down at Character 2.
Character 1Once belonged to a young boy of the city.
Character 1Very foolish one, who wasted many hours slaying dragons and standing to his studies.
Character 2This was yours?
Character 1Yes, it was mine. My father had it made for me.
Character 2Well.
Faint, somber orchestral music and quiet room tone.
15 seconds
An intense interaction taking place in an elegantly decorated room with warm lighting, focusing on an exchange between an older woman and a man, with another woman seen separately in an adjacent space.
Medium close-up of Character 1 leaning close to Character 3, who is facing away from the camera, her face near his ear.
Medium shot of Character 2 standing alone in a hallway, looking off-screen with a somber expression.
Medium close-up of Character 1 sitting upright and looking earnestly at Character 3, whose back is to the camera.
Medium close-up of Character 3 sitting quietly and looking forward, with the shoulder and arm of Character 1 partially visible in the foreground.
Medium close-up of Character 1 gesturing actively with her hands while facing Character 3.
Voice direction. Character 1 speaks with a mature, slightly raspy female voice, starting with a hushed whisper before shifting to an intense normal speaking volume.
Character 1I killed Father.
No speech
Character 1Do you know that there’s a string theory? Do you know what that means? That there are 11, 11 strings.
Character 1It hasn’t been proven yet, but that’s wink. It’s sort of strings, you know? Wink.
A quiet, persistent indoor room tone.
15 seconds
Two men stand in a hallway just outside a brightly lit dressing room, facing a man in makeup inside.
Close-up of Character 1 standing in a hallway, looking slightly off-camera with a serious expression.
Medium shot of Character 2 standing inside a dressing room with vanity lights, smiling and looking toward the doorway, viewed past the shoulders of Character 1 and another man.
Medium close-up of Character 1 and a bearded man with glasses standing together in the hallway, both looking toward the right into the dressing room.
Medium shot from the same angle as shot 2, focusing on Character 2 looking attentively at the men in the doorway.
Character 2 briefly changes his expression to react and maintains a slight smile, still viewed from the perspective of the doorway.
Voice direction. Character 1 has an authoritative, steady older male voice, Character 2 has a soft, slightly raspy male voice, and the background features another male voice.
BackgroundGene, it’s gonna work, it’s gonna work.
Character 1We’re gonna go with it.
BackgroundThank you, Murph.
Character 1Couple of rules, though. No cursing, no raw material. We do a clean show, okay?
Character 2I love Dr. Sally.
Character 1Good, good, good. Someone will come and get you, okay?
Faint continuous room tone.
Results
Four people talk in a quiet living room. Each shot follows a different character.
Character 1, an older man, looks down from a leather chair.
Character 2, a boy in a green hoodie, sits on the couch.
Character 3, a woman in a brown cardigan, looks to her right.
Character 4, a man in a dark blazer, gestures slightly from the couch.
Character 1Do you know Nina?
No speech
Character 2She was my teacher last year.
No speech
Character 3Nina Cabot.
Character 4His homeroom teacher. From last year.
Character 3When he was in the low class.
Découpage
Four people gather around a glowing table in a dim, makeshift lab.
Character 1, a young blond man, watches Character 2 lean over the table.
Character 3 stands beside Character 4, a sheriff. Character 2 is seen from behind.
Character 2 looks down, with Character 1 partly in view.
Character 3 looks intently off-camera.
Back at the table, Character 2 looks toward Character 1, who reacts.
Character 2an artificially intelligent being.
Character 3Then why keep it secret? I mean, this has to be one of the greatest scientific breakthroughs of the modern age.
Character 2So is the atomic bomb. Doesn't mean they're ready for it. Truth is,
Character 2I didn't see if Callister could assimilate.
Character 1So?
Directorial Lip-sync Mechanism · Speaker Assignment
The scene and dialogue stay fixed; only the intended speaker changes.
Matched generations
Character 1 → Character 2 → Character 1
Character 2 → Character 1 → Character 2
Character 1 speaks from 0–4s, Character 2 from 4–10s, and Character 1 from 10–15s.
Character 2 speaks from 0–4s, Character 1 from 4–10s, and Character 2 from 10–15s.
Long-form Results
Four selected minute-long conversations with complete screenplay breakdowns.
Directorial Lip-sync Mechanism · Off-Screen Dialogue
Selective routing keeps the visible listener quiet while preserving the audiovisual context needed across shots.
Full 15-second sequence
No listener-specific routing
Same checkpoint, separate run
Audio-to-video interaction disabled in Shots 2 and 4
Selective listener routing
Character 1 begins “She woke up and goes…” off screen while Character 2 remains visible. Our selective routing keeps the listener's facial motion consistent with the shot.
At the off-screen cue in Shot 2 (5.8–6.5 s), the listener still moves his mouth in the prompt-only rerun.
In Shot 2 of the Skip A2V result, the visible listener makes spurious speech-like mouth movements during off-screen dialogue. The character’s appearance also varies between Shots 2 and 4, reducing cross-shot character consistency in this example.
[VISUAL] [global caption] Two men are engaged in a humorous interaction indoors in a dimly lit room. [character1] Older male, brown hair, full beard, wearing wire-rimmed glasses, a light tan jacket, and a blue button-down shirt. [character2] Younger male, short blonde hair, wearing a grey t-shirt. [per shot caption] [shot 1] From 0 to 2 seconds, A medium close-up of [character1] smiling warmly and reacting while looking slightly off-camera. [shot 2] From 2 to 6.5 seconds, A close-up of [character2] leaning back and laughing with his eyes closed. [shot 3] From 6.5 to 12.5 seconds, A medium close-up of [character1] as he reacts, smiles, touches his head, rests his face on his hand, and laughs. A yellow filing cabinet is visible in the background. [shot 4] From 12.5 to 15 seconds, A close-up of [character2] continuing to laugh heartily with his eyes closed. [SPEECH] [global caption] [character1] has a mature, expressive male voice. [character2] has a younger male voice. [00.0-02.2][character1] One night, it was so loud, it woke the dog up. [02.2-05.8] <NO SPEECH> [05.8-09.2][character1] She woke up and goes, Was that you? See, I didn't have the heart to tell. [09.2-10.4][character2] Oh, God. [10.4-11.9][character1] She woke herself up. [11.9-15.0] <NO SPEECH> [SOUND] [global caption] None
Case 128
Standard generation
Off-screen dialogue instructions
Structural ablation
Selective listener routing
The screenplay keeps the older man speaking off screen as the camera cuts to the younger listener.
The younger man replies off screen while the camera stays on the older listener.
[VISUAL] [global caption] Two men are having a serious interaction in a dimly lit room with a warm-toned lamp visible in the background. [character1] An older Black man with a graying afro and mustache, wearing glasses, a white collared shirt, and a dark vest. [character2] A younger White man with short blond hair and facial stubble, wearing a light gray t-shirt. [per shot caption] [shot 1] From 0 to 3 seconds, A close-up of [character1] looking downward, moving his head slightly as he reacts. [shot 2] From 3 to 6.5 seconds, A close-up of [character2] looking slightly off-camera with a serious expression while interacting. [shot 3] From 6.5 to 11 seconds, A close-up of [character1] looking downward in the dim light, nodding slightly in silence. [shot 4] From 11 to 15 seconds, A continuation of the close-up on [character1] as he remains focused downward, changing his expression. [SPEECH] [global caption] [character1] has a low, calm older male voice, while [character2] has a steady younger male voice. [00.0-03.9][character1] Attrition. It's when you regret your sins, but not for love of God. [03.9-06.7][character2] Unlike because someone's sticking a fucking gun in your face. [06.7-07.4][character1] Mm-hmm. [07.4-10.6] <NO SPEECH> [10.6-13.0][character1] No fainting for totally unrelated victims. [13.0-14.1][character2] Yeah. [14.1-15.0][character1] And no... [SOUND] [global caption] Quiet room tone with a faint continuous hum.
Paper
In cinematic storytelling, sound and visual transitions do not always coincide in lockstep: dialogue frequently bridges across cuts to reveal reactions or introduces new scenes via J- and L-cuts, while deliberate silences pace the scene. Existing audio-visual generative frameworks, however, lack explicit mechanisms to coordinate independent visual and acoustic timing over extended horizons. As narrative duration and shot complexity scale, these systems struggle to maintain long-range dialogue pacing—often hallucinating speech during intended silences or erroneously driving mouth movements on silent listeners. To enable fine-grained cinematic control, we present DÉCOUPAGE, a screenplay-driven framework for long-form, multi-shot, and multi-character joint audio-video generation. Rather than coupling modality boundaries by default, DÉCOUPAGE uncouples generation into independently bounded visual and acoustic tracks governed by persistent character identities. DÉCOUPAGE builds on a joint audio-video diffusion model with dual-schedule temporal cross-attention, incorporating a directorial Lip-sync mechanism that selectively decouples speech conditioning from visual facial dynamics during off-screen dialogue. On multi-shot conversational benchmarks, DÉCOUPAGE achieves precise control over shot transitions, audio timing, and visibility-aware lip synchronization, directing 15- to 60-second screenplay-driven audio-video generation with multimodal coherence.