vllm-router
Subscription: Developer
Active
Subscription: Developer
A high-performance Rust-based load balancer designed for vLLM inference servers, supporting multiple routing algorithms and prefill-decode disaggregation to efficiently distribute requests across GPU workers for large language model serving.