HPE is giving its Alletra Storage MP X10000 another job: holding inference state that would otherwise be evicted from expensive GPU memory and recalculated later. In testing validated by Kamiwaza using NVIDIA H200 GPUs, the Kamiwaza AI platform, and HPE Alletra MP X10000, the solution achieved up to 20× faster time to first token and 17× higher effective inference throughput versus running without KV cache offload. The GPUs did not become cleverer. They had less repeat work to do. The results move a storage decision directly into the discussion about AI economics. Model pricing gets plenty of attention. Paying expensive accelerators to process the same context repeatedly deserves some too. Especially as enterprises move from chatbots answering individual questions to agents that can turn one request into a small committee’s worth of activity. Your agents have a repetition problem
Agentic workflows repeatedly process documents, system instructions and conversation histories. One agent retrieves information, another evaluates it, a third checks the result, and subsequent steps revisit material the system has already processed. Some repetition is necessary. Recalculating everything is not. KV cache retains intermediate attention state, allowing compatible requests to reuse earlier computation. Keeping that cache available can reduce the work required before a model starts producing its next answer. GPU memory, unfortunately, is neither infinite nor priced as though it were. Long contexts and concurrent sessions compete for that memory. When reusable cache is evicted and later needed, the inference engine may have to rebuild it. More GPU work, more waiting, and no additional insight to show for either. HPE Alletra MP X10000 provides a shared tier for offloaded cache, while RDMA for S3 accelerates the storage data path. The proposition: retrieve reusable context rather than make the GPU calculate it again. Compute once. Reuse often. An unremarkable principle, until the alternative appears on the infrastructure bill. Twenty times faster, with the denominator attached
The Kamiwaza-validated testing used NVIDIA Nemotron 70B and more than 30 concurrent agents under large-context, multi-agent inference. That matters because this is precisely the sort of real-world workload where repeated context and memory pressure can make cache reuse valuable. Several agents consulting overlapping material create a different opportunity from unrelated, short prompts with little context in common. Those headline gains are workload-specific. Context length, concurrency, cache hit rates and transfer overhead determine the return. Twenty times faster is a test result, not a universal entitlement. Buyers should examine the comparison as carefully as the multiplier: the same workload, with and without KV cache offload, and enough concurrency to represent production conditions…