Projects
Open infrastructure for accelerator-centric AI systems, organized by the layer and workload they address.
Ecosystem Architecture
(vLLM, SGLang, LMCache, TensorRT-LLM, Agent Applications)
Workload-Aware Storage
β’ KV cache management
β’ Multi-tier architecture
β’ Prefix caching
β’ Control + data path
Direct I/O Optimization
β’ GPU Direct Storage
β’ Eliminates phony buffers
β’ POSIX + io_uring
β’ Data path focus
(NVMe SSDs, distributed storage, filesystems)
Tutti and Phoenix are complementary projects within the xPU-IO ecosystem, exploring accelerator-centric storage from different system scopes and design points. They address different aspects of the storage challenges for AI workloads and can be adopted independently based on specific infrastructure needs.
Tutti
GPU-Centric KV Cache Storage
Tutti addresses workload-aware KV cache management for long-context LLM serving, with a focus on intelligent multi-tier caching (HBM/DRAM/NVMe) and prefix sharing optimized for Agent applications. It provides control-path involvement and data-path optimization for efficient cache storage.
Layer & Focus
- Layer:Workload/Cache Management (Application-level)
- Focus:KV cache strategy, prefix caching, workload-aware storage
- Interface:Cache API for LLM inference frameworks
- Workload:Long-context LLM serving, Agent applications
Phoenix
GPU Direct Storage I/O Stack
Phoenix focuses on optimizing the direct storage-to-GPU I/O data path, eliminating CPU-mediated intermediate buffers (phony buffers). It enables direct DMA transfer between NVMe SSDs and GPU memory, achieving near-theoretical bandwidth with POSIX interface compatibility and cross-vendor GPU support.
Layer & Focus
- Layer:I/O Infrastructure (Kernel-level storage stack)
- Focus:Direct storage β GPU data movement, phony buffer elimination
- Interface:POSIX file I/O, io_uring acceleration
- Support:Cross-vendor (NVIDIA, AMD, MuXi), cross-platform storage