DÉCOUPAGE
Directorial Audio-Visual Control for Long Multi-Shot Video Generation
- 1 The Hong Kong University of Science and Technology
- 2 Ant Group
- 3 Zhejiang University
Generate a scene from a screenplay. Specify when the camera cuts, who speaks, and what the audience hears across shots.
How it works
DÉCOUPAGE takes a screenplay with shot descriptions, dialogue, and ambient sound. It uses separate timelines for the image and the soundtrack, so a line can continue across a cut to the listener.
- Visual direction
- Set the timing of cuts and describe the framing, characters, and actions in each shot.
- Audio direction
- Assign lines to characters and specify when they speak, when they pause, and what else is audible.
- Lip sync control
- Selective audio-to-video routing reduces unwanted mouth movement when the speaker is off screen.