Tag: lmcache

As Agents become mainstream, inference contexts have grown to 1M tokens. Phoenix provides a high-performance direct storage backend for LMCache, reducing average TTFT by 5.6× under high concurrency and boosting system QPS by over 2×.

随着Agent成为主流，推理要处理的上下文已经增长到1M级别。Phoenix为LMCache提供了一个高性能直通存储后端，在生产trace回放中把高并发下的平均TTFT压低了5.6×，推理系统QPS提升2x以上。