AMD’s latest sales pitch: Up to 99 percent of the performance with half the DRAM, provided you’re willing to buy a Mext license alongside your shiny new Epyc server. This is made possible by using AI models to load memory pages in and out of cheaper NVMe storage before your operating system or apps ever know they need it — kind of like SWAP or a Windows page file, but smarter. AMD acquired the tech from Mext back in June, and according to former CEO Gary Smerdon — now CVP of business development at AMD — the tech could cut enterprise server costs by as much as 83 percent by replacing costly RDIMMs with flash storage. Of the initial 145 systems the software has been deployed on so far, he estimates customers have already saved $5 million in avoided DRAM costs. "Conservatively, 60, 70, even 80 or 90 percent for very large memory systems, of the cost of a server can be in memory," he explained. Everyone’s favorite buzzword to the rescue
Even in systems with high memory utilization, where (say) 95 percent of the memory is "in use," a lot of it isn’t actively being used, he explained. He cited a Meta research paper that showed that between 50 and 80 percent of memory pages hadn’t been touched in five minutes, ten minutes, or even longer. "Five minutes to Epyc is an eternity." Mext technology aims to offload those infrequently accessed pages to flash. The challenge, Smerdon explains, is that while flash may be 50x cheaper per bit, 30x lower power per bit, it’s also 500x slower. This is why most folks, when provisioning VPCs or servers, generally avoid swapping out to disk wherever possible: it kills application performance and sends service level agreements out the window. So for Mext’s tech to work, it needed to ensure that any pages offloaded to storage were back in DRAM before they’re needed. Kind of like running an errand in the middle of the workday – so long as you’re back before your boss needs you again, they’ll never know you left your desk. The problem was that the "R" in DRAM stands for random. Predicting which pages the system was going to need at which time isn’t exactly trivial. The solution? (Brace for it.) "We realized we could use AI to monitor the system and proactively push those pages back into DRAM, so that as far as the application or the operating system knew, they were there the whole time," Smerdon said. In other words: "We transparently make flash appear as DRAM." So, "instead of buying two terabytes of DRAM for your Epyc-based server, you could buy one terabyte of DRAM, and then the next terabyte comes with Mext." To do this, AMD’s Mext employs a number of machine learning algorithms. "The workhorse model is a transformer derivative model, just like ChatGPT or Claude or the other famous models, but instead of predicting, for example, in the case of text, what word is next, what we’re doing is predicting what page of memory is going to be required next," Smerdon explained. So long as the pages are in DRAM before they’re needed, the pages don’t even need to be in the right order, he added…