Search. Pull. Build.

The Triton Inference Server provides an optimized cloud and edge inferencing solution with vllm backend

vLLM-Omni extends vLLM to serve omni-modality models, covering text, image, video, and audio, through an OpenAI-compatible API with GPU acceleration.

vLLM is a high-throughput and memory-efficient inference engine for Large Language Models (LLMs). It provides an OpenAI-compatible API server for production LLM deployments with GPU acceleration.