After long speculation and rumors, arm has finally revealed its first hardware CPU, branded as the arm AGI CPU. The company is selling the CPU for revenue to customers, and it is expected to ship later this year. It is an arm-designed production silicon-based CPU, purpose-built to scale compute for the agentic AI era.
Every AI token request needs a CPU to process it. Let’s talk about specs: the dual-chiplet 3nm standard packaging brings up to 136 Neoverse V3 cores. The company showed data with a 300W TDP solution, but since the chip offers variable TDPs from 250W to 430W, each customer can choose to go over or under this number.
Each Neoverse V3 core has 2MB of L2 cache and comes on two side-by-side chiplets. The maximum frequency is 3.7GHz. Each chiplet comes with memory and I/O on the same die, enabling sub-100ns memory latency.
Regarding the packaging, I was on a quest to find out if it is CoWoS, since the chip is manufactured at TSMC. It is not. It turns out that one doesn’t need anything that fancy; it uses MCM (Multi-Chip Module) side-by-side packaging.
The I/O is optimized for composable AI systems and comes with 96 lanes of PCIe Gen6, CXL 3.0 for memory expansion, and more, as well as AMBA CHI extension links. The system supports up to DDR5 8800 memory with 6GB/s of memory bandwidth per core.
Another quest at the arm Everywhere event was to find out which TSMC 3nm process was used. The answer is N3P. It will be available as either an air-cooled or water-cooled rack solution. The air-cooled version comes as a 36kW Open Rack V3 with 30x 2-node 1U servers. The total core count brings it to 8,160 performant CPU cores and 180TB+ of low-latency memory.
The liquid-cooled version is a 200kW open-rack solution with 42x 8-node 1U servers. This alternative cooling method supports a core count of up to 45,696 performant CPU cores and 1PB+ of low-latency memory.
arm claims advancements against an unknown x86 solution, most likely of the AMD EPYC 9005 “Turin” class. These numbers should be taken with a grain of salt, especially as Intel is about to release a high-density E-core solution codenamed Clearwater Forest, and AMD has its own future chips in the works.
The arm AGI CPU development partner list includes Cerebras, Cloudflare, F5, Meta, OpenAI, Positron, Rebellions, SAP, and SK Telecom. Boards and rack-size solutions are coming from ASRock Rack, Lenovo, Quanta, and Supermicro, while Samsung, Micron, and SK Hynix provide memory solutions.
arm claims twice the core density versus x86 solutions, but we are looking for specific ways to back up this claim as new high-density CPUs from both AMD and Intel are set to debut before the end of the year. arm also highlights that it has an ecosystem with 50+ partners. It is a step in the right direction, but it will be interesting to see real-world benchmarks, especially against the Intel and AMD x86 data center competition.
