Tested 16 October 2026 on one RTX 3090. Wav2Lip non-GAN completed this short offline workflow sooner; MuseTalk 1.5 retained crisper visible facial detail. All six runs produced playable outputs. This is one matched research example with three process repetitions per model.
Choose by use case
| Need | What this test supports |
|---|---|
| Short, non-commercial research iterations | Consider Wav2Lip. Lower complete-process wait here, with visibly softer output. |
| Facial detail on this stationary portrait | Consider MuseTalk 1.5. Crisper face and mouth detail in these frames; higher measured wait. |
| Commercial Wav2Lip deployment | The tested upstream code and results have non-commercial restrictions. Check licensing first; the separate commercial Sync Labs service was not tested. |
| Moving heads, profiles, long clips or live requests | No verdict. Those conditions are outside this experiment. See the separate prepared-avatar timing guide. |
Measured process time and memory
| Measure | MuseTalk 1.5 | Wav2Lip non-GAN |
|---|---|---|
| Run 1 | 70.082 s | 11.337 s |
| Run 2 | 54.108 s | 11.134 s |
| Run 3 | 66.452 s | 10.984 s |
| Median process time | 66.452 s | 11.134 s |
| Highest sampled device memory | 7,511 MiB | 4,573 MiB |
| Video frames / video length | 90 / 3.60 s | 88 / 3.52 s |
| Canvas / playback rate | 512 × 512 / 25 fps | 512 × 512 / 25 fps |
| Successful runs / decode checks | 3 / 3 | 3 / 3 |
Timing covers Python launch through exit: imports, model loading, preprocessing, generation and encoding. Setup, checkpoint conversion, transfers and independent validation are excluded. Setup read the models before timing; library/file caches may be warm. These are fresh processes, not cold boots, prepared-avatar runs or live application latency.
GPU memory is whole-device usage sampled every 0.5 seconds, with a 1 MiB baseline and no compute application reported before testing. It can miss brief peaks and is not a minimum VRAM requirement. Both audio/container durations are 3.632 seconds; model-specific audio chunking leaves different video lengths. Native output lengths were retained for timing.
Visual review: subjective
At matched frames 20, 40 and 60 (0.80, 1.60 and 2.40 seconds), MuseTalk shows crisper facial detail and wider mouth openings with visible teeth. Wav2Lip is softer across much of the face; its mouth is closer to closed at 0.80 and 1.60 seconds and opens at 2.40 seconds. Both modify the mouth region. Inspect the paired video with audio before choosing for your material.
Visible articulation does not establish phoneme accuracy. We did not run a blind preference study or validated automatic synchronization/perceptual metric. Native encoders differ, so this experiment does not isolate the cause of softness.
Paired results and original clips
Left: shared input. Centre: MuseTalk 1.5. Right: Wav2Lip non-GAN. First successful run per model. This viewing copy trims to the common 3.52-second video interval and re-encodes the triptych; the original clips below retain native lengths and encoding.
View source video · Listen to source speech · Watch original MuseTalk output · Watch original Wav2Lip output
View all six original clips
Repeated file hashes differ for both models. No seed was supplied. Repetitions measure process-time variation on this sample, not independent scene coverage.
Versions and exact settings
On narrow screens, swipe tables sideways to compare both models.
| Condition | MuseTalk | Wav2Lip |
|---|---|---|
| Candidate | 1.5; model revision 2bcb936e2fddb4d86db4c62fd45b387d0c061571 |
Original open-source non-GAN (SD); not a numbered new release or the commercial API |
| Pinned code commit | 0a89dec45a0192b824e3cf4daf96c239440c5ed8 |
bac9a81e63ecc153202353372e5724b83d9e6322 |
| Native synthesis / precision | 256 × 256; FP16 | 96 × 96; FP32 |
| Controls | batch 2; bbox_shift 0; extra_margin 10; parsing_mode jaw; cheek widths 90/90; fps 25 | pads [0,10,0,0]; detector batch 16; model batch 128; resize_factor 1; static false; smoothing on; crop [0,-1,0,-1]; box [-1,-1,-1,-1]; video fps 25 |
| Environment | Python 3.10.18; Torch 2.0.1+cu118; NumPy 1.23.5; MMCV 2.0.1; mmdet 3.1.0; mmpose 1.1.0 | Python 3.10.18; Torch 2.0.1+cu118; NumPy 1.23.5; librosa 0.9.2; numba 0.56.4; OpenCV 4.9.0 |
Hardware: RTX 3090, 24,576 MiB VRAM, driver 580.173.02, configured power limit 350 W; AMD EPYC 7D12 host with 64 visible logical CPUs. Four OMP/MKL/OpenBLAS threads. Separate environments, sequential processes; no claim of exclusive host ownership. Earlier RTX 3090 articles used another host and power limit, so their timings are not interchangeable.
Matched input: permitted fictional stationary-face video, 512 × 512, 25 fps, 91 frames, 3.64 seconds, silent; identical synthetic English speech, mono 16 kHz, 3.6318125 seconds: “Please bring me a blue mug. We can meet on Friday.” Full source hashes are in the method record.
Run order: Wav2Lip 1, MuseTalk 1, MuseTalk 2, Wav2Lip 2, Wav2Lip 3, MuseTalk 3. Each used a fresh process and result folder without saved avatar/coordinates. All exited 0 and passed complete video/audio decoding on the GPU machine and again after local transfer. Tracked source files remained unchanged.
Fairness limits: shared source bytes and hardware do not equalize synthesis resolution, precision, batches, face detection, audio chunking or encoding. We did not tune either pipeline, equalize compute budgets, randomize the schedule or cool the host between runs. One short stationary scene magnifies startup costs; it cannot establish broad quality, motion performance or a universal best setting.
Setup effort and the actual failure
The pinned Wav2Lip source expects a training checkpoint, but its linked official folder supplied Wav2Lip-SD-NOGAN.pt as TorchScript (145,837,271 bytes). The first setup validation failed with NotImplementedError at the stock loader.
We extracted the unchanged state tensors into a state_dict wrapper accepted by the original inference code. All 352 keys loaded strictly. A CPU comparison on two seeded random test inputs found maximum absolute output difference 0.0 between the supplied TorchScript and reconstructed model. This verifies that conversion test, not all possible model behavior. Original and derived hashes, conversion script and failure log are retained.
Exact checkpoint fingerprints
Original SHA-256: 7ab9249147fd8d7884983567883222b7a86b6f94282354887825dfee7d608999
Derived wrapper SHA-256: d905802fd1efd36993e9c253402b6d79eddb9cc5006784a1b360b464ff01bd30. These identify our retrieved/converted bytes; no independent upstream digest was supplied.
Wav2Lip’s original Python 3.6/Torch 1.1/librosa 0.7 dependency pins were replaced by the recorded compatibility environment above. Both workflows needed model retrieval and dependency preparation. We did not measure human setup effort, so there is no setup-time ranking.
Licence boundary: Wav2Lip authors restrict the repository and results to personal/research/non-commercial purposes, citing the LRS2 training data. Read the current licence statements and commercial-use disclaimer. This research comparison does not test or grant rights to the separate commercial service. Model weights are not redistributed.
Reproducible evidence and related guides
Get the complete evidence archive (ZIP, about 7 MB): original clips, permitted inputs, exact commands, conversion script, environment records and all 12 screenshots. No model weights included.
View guide preparation and publication screenshots
- Comparison draft: original screenshot · version with margin arrows
- Reproducible evidence attached: original screenshot · version with margin arrows
- Published guide: original screenshot · version with margin arrows
View every measurement · Get exact commands, configurations and hashes · Read evaluation notes · MuseTalk dependencies · Wav2Lip dependencies
View identity and saved-evidence screenshots
Continue with video tutorial, photo tutorial, GPU requirements, quality guide, matched examples, prepared-avatar timing, LatentSync comparison and MuseTalk version comparison. MuseTalk pinned source.
2 thoughts on “MuseTalk 1.5 vs Wav2Lip: Tested on RTX 3090”