NVIDIA Run:ai is an AI workload scheduler and GPU infrastructure orchestration layer that provides scheduling and governance features for training and inference workloads. It is intended to increase effective GPU utilization, enforce fair sharing and quotas, and centralize operational control across teams and environments. On Kubernetes, Run:ai provides AI-oriented scheduling capabilities (for example, queueing, quotas, policies, and GPU sharing primitives) on top of native Kubernetes scheduling. (Source: NVIDIA Run:ai product page)
Key functional themes from vendor documentation include:
- GPU resource pooling and policy-based governance across teams/projects.
- Scheduling enhancements for AI workloads (queues, fairness, priority, quotas).
- GPU "fractions" (fractional GPU memory allocation) to run multiple containerized workloads on a single GPU, plus placement strategies to reduce waste.
- Visibility and operations tooling (admin guides, audit logs, troubleshooting log bundles for support).