Nvidia shifts strategic focus to CPUs for next-generation AI

Change in focus comes as a response response to adoption of agentic workflows

Trade

Image: Nvidia

The emergence of agentic AI, capable of performing complex tasks and interacting with the world in new ways, is causing a major shift in computing. While Nvidia’s graphics processing units (GPUs) have long been the driving force behind AI developments, the increasing complexity of agentic AI applications is placing a new emphasis on central processing units (CPUs).

Nvidia is set to unveil its latest CPU developments at its annual GTC conference. That reports CNBC. Those chips play a crucial role in enabling agentic AI workflows. Dion Harris, head of AI infrastructure at Nvidia, said that CPUs are becoming the bottleneck in scaling AI applications, which presents a unique opportunity for growth.

Nvidia’s entry into the CPU market began in 2021 with the introduction of Grace, its first data centre CPU. The next generation is called Vera and is currently in production. Both chips are typically deployed alongside Nvidia’s renowned GPUs.

The explosive demand for GPUs has made Nvidia a household name and the world’s most valuable publicly traded company. However, Nvidia’s recent multi-year agreement with Meta, which sees Grace CPUs being widely deployed, signals a strategic shift towards embracing CPUs as essential components of the AI ecosystem.

This resurgence of CPUs has an obvious cause. The needs of applications for AI are fundamentally changing. While GPUs excel at training and running AI models thanks to their thousands of specialised cores, CPUs are better suited to the general tasks required by agentic AI. These agents often involve coordinating multiple entities and moving huge amounts of data, tasks for which CPUs with their powerful cores are ideal.

Jensen Huang, CEO of Nvidia, highlighted the exponential growth in token generation associated with agentic AI, stressing the need for higher inference speeds. He sees performance per watt as crucial in this changing landscape.

Analyst with Creative Strategies Ben Bajarin described the situation as a “quiet supply crisis” in the CPU market. Leading vendors such as AMD and Intel have reported supply shortages, with delivery times reaching six months and price increases of more than 10%.

Harris argued that Nvidia is coping well with these challenges. The company’s robust supply chain effectively captures the demand for its CPUs.

Nvidia’s CPUs are designed with a specific focus on data processing and agentic AI workflows. Unlike the more common CPUs from Intel and AMD, Nvidia’s Grace CPU features a lower number of cores (72) optimised for passing data to the GPUs.

Harris explains that Nvidia’s approach prioritises single-threaded performance to ensure expensive GPUs are not left idle.

Nvidia’s decision to embrace standalone CPUs comes at a time when hyperscalers such as Amazon, Google and Microsoft are developing their own Arm-based CPUs for data centres.

Nvidia has responded to this trend by opening up its NVLink networking technology for licensing to third parties, allowing seamless integration of CPUs with different architectures with its GPUs.

This platform-independent strategy allows Nvidia to remain competitive regardless of the CPU landscape. As Bajarin puts it, Nvidia’s approach is ‘from A to Z’, offering a comprehensive range of products to meet various AI workloads.

Business AM

Read More: AI Artificial Intelligence Nvidia