Groundbreaking motion adapter enabling any personalized text-to-image Stable Diffusion model into an animation generator.
Motion module weights and inference pipelines completely open source on GitHub, running seamlessly in WebUI and ComfyUI.
AnimateDiff represents one of the most transformative milestones in the history of generative video open-source research. Prior to its arrival, turning personalized text-to-image foundation models into video generators required re-training massive models from scratch for every visual style at astronomical expense. The researchers from CUHK pioneered a revolutionary plug-and-play paradigm: by freezing the pre-trained weights of existing personalized Stable Diffusion models, they trained a generalized Motion Module on video datasets dedicated strictly to learning temporal motion priors. This architecture allows tens of thousands of existing community checkpoints—ranging from stylized anime and photorealistic portraits to cyberpunk and oil paintings—to be instantly animated into buttery-smooth video loops without bespoke retraining. When paired with ComfyUI, ControlNet, and IP-Adapter, AnimateDiff remains the foundational workhorse for generative animators worldwide.