MuseTalk in ComfyUI: Full Setup Guide

If you would rather work with nodes than command lines, the community has built a ComfyUI integration for MuseTalk. This page covers what it is, how to install it, and how to run your first lip sync workflow.

What Is ComfyUI MuseTalk?

ComfyUI MuseTalk is a third party custom node pack built by developer chaojie, which brings MuseTalk’s lip sync model into ComfyUI’s node based interface. Instead of running scripts from a terminal, you load a face (video or photo) and an audio clip through connected nodes and queue the prompt like any other ComfyUI workflow.

A couple of important notes before you start:

This is a community project, not something built or maintained by the original MuseTalk team at Tencent Music’s Lyra Lab. The official MuseTalk repository even says it hasn’t verified or maintained third party integrations, so results and updates depend on the community maintainer.

It’s popular for one clear reason: it lets you combine MuseTalk with MuseV nodes in the same canvas, so a still photo can be turned into a moving, lip synced clip entirely inside ComfyUI, without switching between separate tools.

Requirements

  • A working ComfyUI installation
  • Python environment matching your ComfyUI setup
  • An NVIDIA GPU (4GB VRAM has been reported as workable for lighter runs, more is recommended for longer clips)
  • Around 2 to 3GB of free space for the required model weights

Step by Step Installation

Step 1: Clone the Node into Your Custom Nodes Folder

Open a terminal inside your ComfyUI installation and run:

cd custom_nodes
git clone https://github.com/chaojie/ComfyUI-MuseTalk.git
cd ComfyUI-MuseTalk

Step 2: Install the Required Python Packages

pip install -r requirements.txt

Step 3: Install the MMLab Packages

MuseTalk relies on the MMLab ecosystem for face and pose detection, the same as the standalone version:

pip install --no-cache-dir -U openmim
mim install mmengine
mim install "mmcv>=2.0.1"
mim install "mmdet>=3.1.0"
mim install "mmpose>=1.1.0"

Step 4: Download the Model Weights

You’ll need five sets of weights:

  1. MuseTalk’s trained weights
  2. sd vae ft mse
  3. whisper tiny
  4. dwpose
  5. face parse bisent plus its resnet18 weight

Step 5: Organize the Weights Correctly

This is the part where most first time setups go wrong. Place every file inside your main ComfyUI folder, following this exact structure:

ComfyUI/models/diffusers/TMElyralab/MuseTalk/
├── musetalk
│   └── musetalk.json
│   └── pytorch_model.bin
├── dwpose
│   └── dw-ll_ucoco_384.pth
├── face-parse-bisent
│   ├── 79999_iter.pth
│   └── resnet18-5c106cde.pth
├── sd-vae-ft-mse
│   ├── config.json
│   └── diffusion_pytorch_model.bin
└── whisper
    └── tiny.pt

Note that this folder path is specific to the ComfyUI node and is different from the models folder structure used by standalone MuseTalk, so don’t copy one directly into the other without adjusting the path.

Step 6: Restart ComfyUI

Close and reopen ComfyUI so it can register the new custom node. If the node loaded correctly, you should be able to search for MuseTalk nodes in the node menu.

Loading the Sample Workflow

The repository includes a ready made workflow file called wf.json, along with a reference image (wf.png) showing how the nodes should connect.

  1. Open ComfyUI in your browser.
  2. Use the Load option and select the wf.json file from inside the ComfyUI-MuseTalk folder.
  3. Compare your loaded canvas against wf.png to confirm every node connected as expected.

This gives you a working starting point instead of building the node graph from a blank canvas.

Running Your First Lip Sync Workflow

Once the sample workflow is loaded:

  1. Load your face source. Connect a video or image node with the face you want to animate.
  2. Load your audio. Connect an audio node pointing to the speech clip you want the mouth to match.
  3. Check your output settings. Confirm the result path and format match where you want the final clip saved.
  4. Queue the prompt. Press ComfyUI’s queue button and let the workflow render.
  5. Review the result. Check the mouth timing against the audio, and rerun with adjusted settings if needed.

Turning a Photo Into a Talking Clip (MuseV + MuseTalk)

One of the most common reasons people reach for the ComfyUI version is combining it with MuseV nodes to animate a still photo, rather than only working with existing video. The general flow looks like this:

  1. upload a photo into MuseV nodes first, which generates a moving video from that image.
  2. Take that generated video as the face input for the MuseTalk nodes.
  3. Connect your audio clip to the MuseTalk nodes as usual.
  4. Queue the prompt to produce a final clip where the photo now moves and its lips match the audio.

This two stage approach (MuseV for motion, then MuseTalk for lip accuracy) is the same pipeline recommended in MuseTalk’s own documentation, just wired together as ComfyUI nodes instead of separate command line steps.

Troubleshooting Common Issues

Node doesn’t appear after installation

Confirm you cloned the repository directly inside custom_nodes, not a subfolder within it, and that you fully restarted ComfyUI afterward.

Model not found errors

Recheck Step 5 closely. The folder path for ComfyUI (ComfyUI/models/diffusers/TMElyralab/MuseTalk/) is easy to get wrong if you’re used to the standalone MuseTalk folder layout.

Workflow runs but the mouth barely moves or looks off

Try adjusting your source video’s frame rate closer to 25fps before feeding it in, since that matches what MuseTalk was trained on.

Out of memory during rendering

Shorten your input clip, lower the output resolution, or close other GPU heavy applications before queuing the prompt.

Something broke after a ComfyUI update

Since this is a community maintained node and not an official Tencent Music project, compatibility with the newest ComfyUI releases can lag.

Check the repository’s issues page for similar reports before assuming your setup is at fault.