SGLang

SGLang is an open-source LLM/VLM inference serving framework and Python-embedded DSL that provides low-latency, high-throughput inference, controls for multi-step generation workflows (structured outputs, tool-calling, multi-turn chat with prefix reuse), and a multi-component stack including a runtime, model gateway, and diffusion optimizations.