xPU.IO: Direct StorageInfrastructure for AI Acceleration
Building high-performance GPU-centric storage solutions for LLM inference and AI workloads
xPU.IO is an open-source community initiated by Tencent DCPD, focused on GPU Direct Storage for AI applications. We build production-ready infrastructure that eliminates CPU bottlenecks and unlocks the full potential of NVMe SSDs for AI workloads.
The AI Storage Challenge
KV Cache Capacity Crisis
As Agent applications become mainstream, inference contexts have grown to 1M+ tokens, with single session KV Caches reaching tens of GB. HBM and DRAM capacity is limited and expensiveβlarger storage media are needed.
Storage Speed Bottleneck
Traditional storage paths require data to traverse CPU and host memory, causing bandwidth loss exceeding 60% and severe latency impact on inference performance.
Traditional Path vs GPU Direct Storage
Our Approach: GPU-Centric Storage Stack
xPU.IO leverages GPU Direct Storage technology to enable storage devices to transfer data directly to GPU memory, eliminating CPU and host memory intermediate overhead.
Extreme Performance
Achieve up to 96% of NVMe theoretical bandwidth with direct GPU access, compared to 40% with traditional CPU-mediated paths.
5.6Γ faster TTFT
Massive Capacity
Scale KV cache storage to 100TB+ per machine using NVMe SSDs, enabling 1M+ token contexts at a fraction of HBM/DRAM cost.
100TB+ per machine
Low Cost
Reduce storage cost by 10-100Γ compared to HBM/DRAM while maintaining high performance for AI inference workloads.
10-100Γ cost reduction
How xPU.IO Achieves This
Our Projects
xPU.IO develops GPU-centric storage infrastructure for AI workloads, from low-level I/O stack to high-level cache management.
Multi-tier KV cache management system that leverages NVMe SSDs as the primary storage tier, supporting 256K+ context lengths with efficient prefix caching.
- 256K+ context support
- NVMe-backed storage tier
- Prefix cache optimization
- Multi-tier cache management
Open-source GPU Direct Storage middleware that eliminates CPU bottlenecks by enabling direct data transfer between NVMe SSDs and GPU memory, with POSIX interface and cross-vendor support.
- POSIX interface compatibility
- io_uring acceleration
- Cross-vendor GPU support (NVIDIA, AMD, MuXi)
- 25 GB/s throughput
Integration Showcase
xPU.IO integrates with leading AI frameworks and inference engines.
Inference Engines
GPU Vendors
Storage Backends
Research
Our work is grounded in peer-reviewed research published at top systems conferences.
Phoenix: A Refactored I/O Stack for GPU Direct Storage without Phony Buffers
SC'25 - International Conference for High Performance Computing, Networking, Storage, and Analysis
Tutti: GPU-Centric KV Cache Storage for Long-Context LLM Inference
arXiv preprint, 2026
Join the Community
xPU.IO is an open-source community. Contribute code, report issues, or join discussions.
Ready to Get Started?
Explore our projects, read the documentation, or dive into the research.