DÉCOUPAGE

Directorial Audio-Visual Control for Long Multi-Shot Video Generation

  • Yifan Wang1,2
  • Hao Ouyang2
  • Qiuyu Wang2
  • Yihao Meng1,2
  • Wen Wang3
  • Zeyue Tian1
  • Yichong Lu1,2
  • Hanlin Wang1,2
  • Qingyan Bai1,2
  • Zichen Liu1,2
  • Yujun Shen2
  • Qifeng Chen1
  • 1 The Hong Kong University of Science and Technology
  • 2 Ant Group
  • 3 Zhejiang University

Generate a scene from a screenplay. Specify when the camera cuts, who speaks, and what the audience hears across shots.

The Trumpet Deal60 seconds · 8 shots

How it works

DÉCOUPAGE takes a screenplay with shot descriptions, dialogue, and ambient sound. It uses separate timelines for the image and the soundtrack, so a line can continue across a cut to the listener.

Visual direction
Set the timing of cuts and describe the framing, characters, and actions in each shot.
Audio direction
Assign lines to characters and specify when they speak, when they pause, and what else is audible.
Lip sync control
Selective audio-to-video routing reduces unwanted mouth movement when the speaker is off screen.

Generated scenes

See all results
Across the Glass60 seconds · 7 shots
Conversation on a Moving Train15 seconds

Compare the results

Baseline comparison Two scenes, four generators, and the shared screenplays. Lip sync control Watch the visible listener during off-screen dialogue.