AI has become a juggernaut, driving technical strategy in the tech sector and outside of it. With the worldwide server market growing 30 percent in a single year to hit $122. 6bn in the first quarter of 2025, IDC now feels safe recategorizing AI-driven investment "from cyclical to durable". The analyst company expects inference to account for over 70 percent of computing demand by next year, adding that it’s growing fastest at the edge. Why enterprises are building their own
All this investment in AI infrastructure might once have come mainly for hyperscalers, but that’s changing quickly. Adoption is spreading quickly beyond public cloud service providers into enterprise environments, IDC adds. They’re eager to run agentic AI in-house. There are plenty of reasons for companies to own their own AI infrastructure rather than offloading it to the cloud. One of these is data control, especially in regulated sectors such as finance, healthcare, manufacturing, and the public sector that have sovereign data requirements. Another revolves around the need for predictable costing from tokens delivered on tunable in-house systems. So what should companies look for in a supplier? Delivery speed will be a powerful driver for many who are under the gun to deliver AI initiatives for the business, especially during periods of constrained memory and CPU supply. Life cycle services will also come near the top of the list, for companies that need help managing everything from the power through to the reasoning LLM models and the workloads that use them. That need for expertise is unlikely to stop; companies will undergo changes in AI strategy as they get to grips with a still-evolving technology. From the desk to the datacenter
ASUS expects this AI surge to continue in the datacenter, but elsewhere too, specifically in edge locations (especially industrial use cases) and at developers’ desks, as the applications that will power the enterprise get made. It wants to be top of peoples’ list for hardware and services at all these places along the AI continuum. At NVIDIA’s GTC show in Berlin on Oct 21-22, it announced a full-stack AI offering that it’s calling its Enterprise Agentic AI Solution . It’s three-layer affair, featuring:
ASUS AI Server and Infrastructure: Its selection of AI servers, including several new units. ASUS AI Software Stack: Its software for integrating and controlling AI workflows. ASUS AI HUB: A services operation that ties it all together. These bring together some existing assets that ASUS had for enterprise AI users, along with some new ones. The time is right for an end-to-end offering with new hardware because of the way the market is shifting, say ASUS execs. "ASUS servers help enterprises build high-performance agentic AI infrastructure capable of supporting both complex task execution and large-scale AI agent deployment, accelerating enterprise adoption of agentic AI," says Paul Ju, senior vice president and head of infrastructure solution BG at ASUS. While generative AI initially spiked demand, the advent of agentic AI has sent compute demand into overdrive. Ju explains that the focus is expanding from the GPU. "Historically, AI servers have been optimized primarily around GPU compute. But as enterprise AI shifts toward agentic AI, workloads increasingly revolve around a continuous ‘agent loop’ that is heavily CPU-dependent," he says. That loop takes in task planning, code execution, external tool invocation, and reflection or correction work. The CPU’s increased role in this execution work is creating bottlenecks in conventional AI servers, he adds. The first such sticking point is single-thread performance, which leaves GPUs idling while they wait for CPUs to complete. The second lies in memory bandwidth. When this is managed on a per-core basis, it can slow things down when many agents run at once. The multi-die design of many CPUs also penalizes them in AI environments, because accessing data across dies introduces latency, Ju argues. CPUs with cores and cache on different dies suffer latency when data has to cross between them. Maintaining cache coherency exacerbates that problem. Finally, there’s a security risk: a lack of hardware level sandbox isolation on some CPUs make it possible for prompt injection to reach core systems, he warns. Customers also bring their own challenges as they grapple with different applications for AI. Enterprises evaluating AI server options have to match their server platforms to the projected AI workload profiles, Ju adds. Training a supervised learning model to batch-classify insurance claims takes different technical resources to running a low-latency reasoning employee chatbot. What ASUS took to Berlin
ASUS wants to help customers solve problems like these wherever they come up. With this in mind, the company had plenty of goodies for onlookers at GTC…