Wav2Lip Studio

Wav2Lip Studio is an extension for the Stable Diffusion WebUI Automatic1111 that generates lip-sync videos from a chosen video and speech file. It is built for AI video creators who want higher-quality Wav2Lip output than the base tool produces, using Stable Diffusion-based post-processing to refine the result. A separate standalone version of the tool is also available outside Automatic1111.

A user installs the extension into Automatic1111 and places the required model weights in the correct directories, then chooses a video and speech file in the UI. The extension generates a low-quality Wav2Lip video from the inputs, enhances it with a chosen restoration model, creates a mask around the mouth, and overlays the enhanced mouth onto the original video before using FFmpeg for final post-processing.

Wav2Lip Studio supports experimental face swap through FaceFusion integration, text-to-speech generation through the bark model in 13 languages with adjustable gender, speaker and temperature settings, and a GFPGAN or CodeFormer restoration model with an adjustable fidelity value to balance quality against facial consistency. Controls include padding, resize factor, mouth-only tracking, mask dilation and erosion, and a resume option to adjust the mask without regenerating the face swap and Wav2Lip steps.

Wav2Lip Studio's extension code is released under the Apache-2.0 license and requires the latest Stable Diffusion WebUI Automatic1111 plus FFmpeg accessible from the command line; on Windows, Visual Studio with Python and C++ packages is needed for face swap. The tool requires a face visible in at least one frame of the video, bark audio generation is limited to 14 seconds per segment without using a split marker, and dlib must be installed differently on Mac.

Use cases

• Creating a lip-synced talking-head video from a recorded speech file
• Swapping a face in a video before lip-syncing it
• Generating narration audio from text with a chosen voice and language