vLLM Production Stack
The vLLM Production Stack project provides a reference implementation on how to build an inference stack on top of vLLM with LMCache. It allows you to scale from single vLLM instance to distributed vLLM deployment without changing any application code, monitor the metrics through a web dashboard, and enjoy the performance benefits brought by request routing and KV cache offloading.