Tencents open-source framework generating photorealistic portrait animation from audio and pose sequences.
Open-sourced by Tencent and researchers, synthesizing high-fidelity portrait animation guided by audio and pose skeletons.
AniPortrait is a cutting-edge open-source framework for synthesizing high-quality, photorealistic portrait animation, developed by Tencent researchers in collaboration with academic laboratories. While earlier talking-avatar models struggled with jittery facial boundaries and plastic-looking expressions, AniPortrait achieves remarkable visual naturalism through a modular two-stage pipeline. In the first phase, an Audio2Lmk module extracts dense 3D facial landmarks and head pose trajectories from audio signals. In the second phase, a robust diffusion model combined with motion cross-attention modules synthesizes temporally smooth, high-fidelity video sequences. It delivers synchronized, phoneme-accurate lip movement alongside organic head rotations, subtle breathing, and natural shoulder posture. AniPortrait flexibly supports both audio-driven and pose-driven modes, making it an invaluable asset for game developers, digital humans, and generative cinema.