Skip Navigation (Press Enter) Skip to Main Content (Press Enter)

DPUs: Dedicated Data Chips Tackle Tedious Tasks

Data processing units (DPU) offer specialized processing for networking, storage, and security, lowering costs while boosting performance per watt

Eddie Ramirez Headshot
Posted on 14th October 2021 By Eddie Ramirez, VP, Infrastructure Business, Arm
Cloud Computing
DPUs: Dedicated Data Chips Tackle Tedious Tasks

Data Processing Units (DPUs), also known as Infrastructure Processing Units (IPUs), are made to handle the behind-the-scenes work of cloud computing: exacting and time-sensitive security tasks like packet inspection and hypervisor management so video streams can flow and artificial intelligence (AI) algorithms can unearth unexpected trends.

And that’s exactly why specialized processors are rapidly becoming an exciting segment in the industry. Network, storage, and security tasks can now consume as much as 30 percent of a server’s CPU cycles for the resources required by an instance.

A dedicated processor optimized for these tasks–one that is better than them, in fact, than a central processing unit (CPU)–creates an opportunity for customers to better focus their CPU cycles towards paying workloads instead of infrastructure tasks, yielding TCO benefits in power and capex.

Some notable announcements include:

DPUs and IPUs are becoming the control point of the cloud network and that is enabling some profound improvements in security and composability.  From a security standpoint, these technologies are the gold standard in system security.

Whether you are talking about booting secure and attested software, threat detection or deep packet inspection of all incoming network traffic, none of that is practical at cloud scale without DPUs.

DPUs: A Decade of Innovation

Efforts to develop DPU-like systems to handle network, security and storage functions began around a decade ago. In 2014, Amazon Web Services, for instance, began to work with Annapurna Labs, a startup designing Arm-based chips, to develop a system for accelerating storage tasks. It bought Annapurna the following year and in 2017 debuted the Nitro System for managing a broad menu of tasks like hypervisor management, virtual private clouds, elastic block storage and high-performance networking. (The Nitro System is comprised of three main parts: the Nitro Cards, the Nitro Security Chip, and the Nitro Hypervisor.)

Since then, Nitro has become part of the foundation of modern EC2 instances, AWS Outpost, Mac Instances and other services.

Not only has the Nitro System improved customer experiences, Amazon CTO Dr. Werner Vogels says it gives AWS more bandwidth to innovate. Since launching Nitro, the number of instances available on AWS has grown by 4x.

“Whether you chose to look at something as a problem or an opportunity can have large effect on how you deal with it. We have chosen to look at the limitations of the traditional hypervisor as an opportunity to create a completely new architecture,” he writes in his All Things Distributed blog. “Now with the Nitro System, our customers enjoy better performance, enhanced security, and a broader set of instance types to choose from. And we’re not done yet.” 

Anatomy of a Data Processing Unit (DPU)

DPUs are an evolution of “SmartNICs” like AWS Nitro and FPGAs, which were integrated into cloud operations for offloading administrative tasks. DPUs aren’t offload devices. They are dedicated, co-equal processors that boot their own operating system (OS) independent of the host OS and manage their own memory and storage.

They also add programmable packet processing (with P4, for example) to a base of secure boot, high-speed networking and performant, low-power general purpose cores. Not only does carving out these tasks into a separate domain improve performance, it expands what cloud providers can accomplish with them. If the CPU is the CEO, the DPU is the COO.

While hyperscale cloud providers are currently the main consumers of these products, the technology will also spread to enterprise data centers and high-performance computing laboratories. . With Project Monterey, VMWare is moving the ESXi hypervisor onto DPUs and recognizing the benefits of specialized hardware to deploy enhanced virtualized services.

This will enable faster adoption and cloud-native software in enterprise and telco markets. Project Cassini and Arm SystemReady, meanwhile, give developers, hardware manufacturers and their customers a glide path toward interoperability across the stack.

The Next Frontier

The technology will likely percolate to other markets and use cases as well. The AI and machine learning (ML) and security requirements of new applications—think high-end robotics or autonomous driving— could well prompt designers to adopt specialized data processing into their product designs.

Device architecture will also become increasingly complex. NVIDIA’s Bluefield-4, for instance, will contain multiple CPUs and GPUs to achieve a quadrillion operations per second. At some point, neural processing units (NPUs) could become part of DPUs.

This is a space well worth watching.

Click to read Arm Neoverse: The Next Wave in Digital Infrastructure

Subscribe

Sign up to receive the latest from Arm Blueprint
We will process your information in accordance with our privacy policy. In subscribing to Arm Blueprint you agree to receive a monthly bulletin email of new content, as well as one-off emails for launch blogs, executive communications and new Arm reports.