sglang-router python library

sglang-router

Subscription: Developer
Last updated 20 days ago
Active
Subscription: Developer

A high-performance Rust-based load balancer for SGLang inference servers, providing multiple routing algorithms including round-robin, random, and cache-aware strategies with prefill-decode disaggregation support for LLM serving workloads.

pip install sglang-router==0.3.2
  • 0.3.2
    Linux
    Last updated 20 days ago