Industrial-grade high-throughput, low-latency open-source LLM inference and serving engine powered by PagedAttention.
Fully open source under Apache 2.0, freely deployable on any cloud or local GPU cluster with zero licensing costs.
vLLM is the global industry standard for high-throughput, low-latency LLM serving and inference. Developed by researchers at UC Berkeley, it invented PagedAttention—an innovative memory management algorithm managing Key-Value cache like virtual memory in operating systems—achieving up to 24x higher serving throughput than standard baseline frameworks.