Overview

Rethinking the Datacenter for the Agentic AI Era

AI is driving data centers toward specialized, workload-optimized infrastructure that emphasizes power efficiency, scalability, and performance. Arm delivers the CPU foundation for AI data centers, integrating seamlessly with accelerators to orchestrate AI agents, process data and support scalable AI workloads, such as recommendation engines, large language models, retrieval-augmented generation and more. Paired with a robust software ecosystem, Arm compute platform enables hyperscalers to scale AI infrastructure efficiently while improving performance, cost and energy outcomes.

Download eBook
 

Inside the AI Datacenter: Custom Silicon and the Power of the Arm Ecosystem

Hear from Mohammed Awad, head of the cloud AI business unit at Arm, as he explores how AI is reshaping datacenter design, why performance per watt now defines cloud competitiveness, and how the Arm ecosystem is accelerating next-generation custom silicon for the AI era.

Benefits

More Compute, Higher Efficiency, Better Price-Performance

Arm delivers energy-efficient compute that pairs seamlessly with a broad range of AI accelerators—helping you achieve strong performance and efficiency while lowering total cost of ownership.

Up to 8x Faster ML Model Training and 4.5x LLM Inference Performance

Delivered by the NVIDIA Grace Hopper Superchip when training a DLRM model and inferencing GPT-65B model, compared to x86+Hopper systems.1
Up to 168% Better LLM and 53% Better ML Performance

Delivered by AWS Graviton4 processor in LLAMA 3.1 and XGBoost benchmarks compared to x86 alternatives. 2
Up to 2.5x Higher AI Inference Performance

Delivered by Google Axion processor, with 64% cost savings and faster RAG for real-time AI compared to x86 alternatives.3
Up to 1.9x Better LLM Inference Performance with 2.8x Better Price/Performance

Delivered by Microsoft Cobalt 100, compared to x86 alternatives.4
Partners

Enabling Industry Leaders Though Infrastructure Optimized for Real-World Performance

Arm empowers industry leaders to build scalable, AI-optimized cloud infrastructure with computing solutions tuned for real-world AI performance. Designed for performance, power efficiency, and seamless scalability, Arm CPUs are perfectly suited to orchestrate accelerators for the most demanding AI and cloud workloads.

Arm and AWS

Discover how Arm-based AWS Graviton processors are transforming cloud AI with leading price performance and efficiency for AI and cloud-native workloads, now powering AWS Trainium3 UltraServers.

Explore Arm and AWS

Explore how Axion, the first Google Cloud custom Arm-based CPU, is advancing performance and efficiency for AI and cloud workloads.

Explore Arm and Google Cloud
Arm and NVIDIA

Discover how Arm’s power-efficient compute platform has become a key element in NVIDIA accelerated computing platforms, including the Grace CPU family and now Vera CPUs, delivering performance leap in NVIDIA’s rack-level AI solutions.

Explore Arm and NVIDIA
Compute Platform

Powerful Agentic AI Performance with Arm Neoverse

Designed to handle demanding AI workloads efficiently, Arm Neoverse CPUs deliver high throughput and performance per watt—making them ideal AI head node and orchestration engines in AI datacenters. From recommendation engines and language model inference to retrieval-augmented generation (RAG), Neoverse scales across a broad range of agentic AI applications.

Performance

Up to 3x better recommendation model performance on Google Axion vs. x86 5.

cost saving

Up to 2.5x higher AI inference throughput with 64% cost savings compared to x86 alternatives 6.

Ecosystem

Broad hyperscaler adoption and multi-cloud availability.

Explore Arm Neoverse for AI WorkloadsLearn About AI/ML on CPU

Arm Compute Platform for Every AI Workload

As AI progresses from classic machine learning to generative AI and now agentic models, workloads are becoming increasingly compute and power intensive. Meeting these demands requires a shift to purpose-built CPUs which empower AI systems to dynamically match each workload with the right processor, optimizing for performance, power efficiency, and cost.

 

Arm Neoverse CPUs provide a power-efficient, scalable compute platform that integrates seamlessly with GPUs, NPUs and custom accelerators and delivers increased performance, flexibility, efficiency, and scalability.

Explore Heterogeneous Computing Solutions
Software and Developer Tools

Optimize AI Workloads with Arm Software and Tools

Developers need optimized tools to deploy AI quickly and efficiently with little effort. The Arm software ecosystem—including Arm Kleidi libraries and broad framework support—helps accelerate time to deployment and boost AI workload performance across cloud and edge.

Resources

Latest News and Resources

  • NEWS and BLOGS
  • Report
  • Podcasts
  • White paper

Benchmarking Sustainable Datacenter Performance

Independent analysis from Signal65 reveals how Arm Neoverse-based AWS Graviton4 processors consistently deliver superior performance per watt across web, database, and AI workloads—driving greater efficiency and lower total cost of ownership in datacenters.

Learn More
AI in Datacenters

The Dawn of a New Era for Arm in the Datacenter

Industry analyst Ben Bajarin explores how AI is redefining datacenter architecture and why Arm is emerging as a key player in powering scalable, efficient infrastructure for the AI era.

Read Now
Podcast icon
AI in Datacenters

Arm and NVIDIA Redefine AI in Datacenters

Listen to our podcast with NVIDIA to explore how our partnership is transforming enterprise computing.

Listen Now
Podcast icon
AI in Datacenters

The Future of AI Infrastructure with Arm and Industry Expert Matt Griffin

Hear Arm and Matt Griffin, founder of the 311 Institute, discuss emerging AI infrastructure trends, challenges in scaling compute, and how Arm is enabling efficient, sustainable AI from cloud to edge.

Listen Now

Build a Scalable AI Platform from Cloud to Edge

Learn five decisions that help enterprises design a future-ready compute stack. Explore how to embrace heterogeneous compute, unify the software layer, and align infrastructure with business goals to cut latency and scale efficiently across environments.

Read white paper

Key Takeaways

  • Arm enables datacenter transformation from general-purpose platforms to specialized, workload-optimized AI infrastructure built for efficiency and scalability.
  • Neoverse CPUs deliver high throughput, power efficiency, and lower TCO for AI applications including recommendation engines and large language model inference.
  • Arm-based processors from partners like Google, AWS, Microsoft and NVIDIA achieve up to 8x training and 4.5x inference performance gains over x86 systems.
  • Heterogeneous Arm-based infrastructure dynamically matches workloads with CPUs, GPUs, NPUs, and custom accelerators for optimal performance and cost.
  • Arm’s Kleidi libraries, frameworks, and developer tools streamline AI deployment and workload optimization across cloud and edge environments.

Frequently Asked Questions: AI in the Datacenter

What makes Arm ideal for AI in datacenters?
  • Power-efficient performance: Arm Neoverse CPUs deliver industry-leading performance-per-watt, reducing energy costs and improving operational efficiency.
  • Lower total cost of ownership (TCO): Scalable architectures optimized for modern AI workloads help businesses reduce infrastructure spend.
  • Flexible, workload-optimized systems: Arm-based platforms seamlessly integrate with GPUs, NPUs, and custom accelerators to deliver the right compute for every AI task.
  • Trusted by hyperscalers: By 2025, half of compute shipped to top hyperscalers is projected to be Arm-based—underscoring growing confidence in Arm for large-scale AI deployment.
  • Unified AI infrastructure: A mature software ecosystem and broad adoption support seamless integration across diverse compute engines in cloud and datacenter environments

How do Arm-based platforms enhance AI performance and reduce cloud costs across industry partners like NVIDIA, Google Cloud, and AWS?

Arm-based platforms boost AI performance and efficiency at scale:


Together, these innovations enable faster, more cost-effective AI across cloud and hyperscale platforms.

What tools does Arm offer to developers for AI workloads?

Developers can accelerate workloads using:


Stay Connected

Subscribe to stay up to date on the latest news, trends, case studies, and technology insights.

1. NVIDIA GH200 Grace Hopper Superchip Architecture
2. Results are based on 3rd-party evaluations. All tests are conducted by Signal65 and measured
on AWS instances and specifications noted at the time of testing. Full methodology here. Results may vary.
3. Harness the Power of Retrieval-Augmented Generation with Arm Neoverse-Powered Google Axion Processors
4. Accelerate LLM Inference with ONNX Runtime on Arm Neoverse-powered Microsoft Cobalt 100
5. Unpacking Axion: Google Cloud’s Custom Arm-based Processor Built for the AI age
6. Harness the Power of Retrieval-Augmented Generation with Arm Neoverse-Powered Google Axion Processors