vLLM

AI Native

vLLM is an open-source large language model inference and serving engine that implements PagedAttention and continuous batching to improve throughput and memory efficiency and exposes an OpenAI-compatible HTTP server for integration.