The world’s best PC, server and genuinely advanced AI inferencing chips have been delivered by the intellectually augmented innovators at AMD, whose artfully intense efforts to actively advance AI are indisputably paying off, delivering better, more powerful and more affordable tech for all, challenging competitors to do better, and bringing tomorrow’s promises into today’s reality. This COMPREHENSIVE feature has ALL the videos and information AMD shared and that I captured!

With AMD’s new MI-325X AI Accelerator chips more powerful than Nvidia’s H200, with AMD’s Ryzen chips for desktops and notebooks offering 50 and 55 TOPS (trillions of operations per second) of AI power handily beating Intel’s 48 and inexplicably new but slow 36 TOPS chips, Apple’s 38 TOPS in the M4 chip, and Qualcomm’s 45 TOPS with the Snapdragon X Plus and Elite, and with the new 5th-gen EPYC processors offering an absolutely astounding 7:1 efficiency rating compared with the so-called “equivalent” Intel Xeon, as well as Ultra Ethernet technologies that allow data centre and AI GPUs to communicate internally at vastly faster speeds, eliminating bottlenecks, it’s clear that AMD is on a massively advantageous roll, handily winning market share, ever more industry support and ever stronger business and consumer loyalty.  

This is a feature covering AMD’s Advancing AI 2024 event in San Francisco. Here are the jump points to go directly to each section in this long article below:

1. AMD’s announcements – the overview
2. AMD Ryzen Pro 300 AI announcement
3. AMD EPYC 5th-gen announcement
4. AMD MI-325X Instinct announcement
5, AMD’s Pensando networking super speed breakthrough video
6. Major AMD Epyc Q&A briefing session
7. Minor AMD Epic Q&A session
8. Era of Gen AI presentation
9. AMD Instinct Q&A session
10. Dr Lisa Su Final session plus competition winners

The full MUST-WATCH keynote launch is embedded directly below, and it is an absolute Tour-De-Force from AMD, showcasing its most advanced technologies yet, while also promising even better to come in the years ahead, after which is a brief look at how we got here, and then diving deeper into the processors themselves, so let’s start with this incredible keynote!

Here is AMD’s CEO and President, Dr Lisa Su, talking about AMD’s advances in its technologies and AI capabilities on the Bloomberg news channel:

Ok, so AMD was founded on May 1, 1969, less than a year after Intel’s founding on July 18, 1968, and both tech titans were founded before man first set foot on the moon on July 20, 1969.

In the 55 years since AMD’s founding, that moon landing, and AMD’s chips reaching a market leading 55 TOPS, the world is vastly different in technological terms. The AI powering the robots in Isaac Asimov’s first robot novels, published from 1940 onwards, now 84 years ago, is now more real and with us than ever, thanks to generative AI – and it’s just the beginning. 

Gen AI no longer needs to be solely accessed via the cloud and AI chips in data centres, but can work on today’s desktops, laptops and even smartphones in a purely on-device way. Of course, even in 2024, you still get the full force of true Gen AI and machine learning power with help from much more powerful cloud AI inferencing and computing services, but the shift from centralised AI to decentralised AI has well and truly begun, and accessing AI can be done without being online if you want, and even more so if the NPU is further augmented by a separate GPU that is locally connected but still disconnected from cloud AI compute.

Having AI that works entirely on device is now a reality, and with every new chip generation onwards, is only going to get more powerful… but we are here at last! The missing piece of the robot revolution – its brain – can now interact with you generatively, as we’ve seen with ChatGPT and its competitors, and interacting with a computer in a conversational way has never felt more human. It is also no longer the confines of sci-fi robots like C3P0, Chappie, Number 5 is alive, Marvin the Paranoid Android, Commander Data, Wall-E, along with Robby and Rosie the Robots, among many others (cough, T-800, cough).

That said, we’re still in the early stages of the AI revolution with ChatGPT from OpenAI only 23 months old – still not even at the two year mark yet, so we’re truly at the beginning of this incredible revolution, and AMD is one of the companies at the very forefront of it all. 

But sci-fi turning into sci-fact aside, home robots are at least half a decade away or more before they’re affordably in homes working to help humans live better lives, despite the advanced humanoid robots we see from Elon and others, which are still decidedly primitive compared to Asimov’s R. Daneel Olivaw, but there’s every chance an AMD powered Asimovian-positronic AI processor of the future will be powering its brains come 2030 – and in your home, helping YOU as your family’s assistant or even your own personal robot sidekick as we’ve seen in Star Wars and many other movies and TV shows. 

We can foresee this future because of the work AMD has been doing over the last few years with its Zen technology which debuted in the first Ryzen and Epyc processors launched in 2017, ending more than a decade of Intel’s sloppy and lazy advance though the late 2000’s to 2017, where iterative year or year performance improvements and no real competition caused Intel to coast along, stuck in a groove of i3, i5, i7 and Xeon processors that advanced ever-so-slowly. 

Let’s not even talk about Celeron, Pentium Silver/Gold and the long-since abandoned Atom and despicably woeful Core m-series processors that were hopeless ways to lower prices for shoddy low-end PCs – notwithstanding the advanced 12-inch MacBook of 2015 that was almost perfect, but seriously hobbled thanks to the craptacular Core Intel m3 and m5 processors that weren’t worth the silicon their processors were etched upon. 

AMD was the one to announce the first-gen Ryzen CPUs for everyday computing with 8 cores and 16 threads on December 13, 2016, and sold from early May 2017, in a world where Intel’s i7 were stuck for years offering 4 cores and 8 threads. 

AMD’s startling resurgence shocked Intel to life, bringing forth the Core i9 to sale in June 2017 in a move that presumably would have taken even longer had AMD not awoken the giant. This seminal moment was the start of Intel’s ongoing decline which still continues to this day, and even though Intel still sells more CPUs than AMD, after 5 years the tide is turning faster than ever before. 

So, let’s fast forward nearly 8 years on from Ryzen and the first Zen Core architecture, to today and the fifth generation of the Zen platform, and not only has AMD’s share of EPYC powered systems risen to 34% of the server market – a phenomenal achievement – but it’s AI chips for desktops, notebooks and servers are the most advanced in the industry, far surpassing Intel and Qualcomm, especially in the AI TOPS stakes as noted before, while its AI inferencing MI series chips are outclassing and challenging NVIDIA’s H100, H200, and B200 chips for sales, dominance and sheer AI power.

It has given AMD the confidence to predict a global data centre AI market of $500 BILLION by 2028, up from $45B last year.

AMD is an unquestionable leader in the AI space! Where once the conversation was mostly NVIDIA, AMD has become an AI force to be reckoned with, and they’re not sitting still, but have already previewed MI-350X and even MI-400X to come, although with MI-325X now in production for Q4 2024, that’s where all the focus now rightly shines. 

Sure, NVIDIA has its fancy new Blackwell GB200 GPU’s that are absolute beasts, and only available to the select few tech titans at the moment, but with AMD’s original MI-300X able to outperform NVIDIA’s H100 chip and seriously challenge its H200 chip, it’s no surprise to see organisations like Meta, which obviously uses NVIDIA’s chips too, having announced at AMD’s October 2024 event that is has installed 1.5 million Epyc processors, and that all live traffic for Llama 405b runs exclusively on MI-300X. And AMD will soon see its upgraded MI-325X brother being made available by the major manufacturers and hyperscalers for rapid deployment, at prices expected to make customers very happy. 

You can find out specific details of the AMD Ryzen 9000 processors for desktops and Ryzen AI Pro 300 processors for notebooks, with the media release for the AI Pro 300 processors in a media release titled: “AMD Launches New Ryzen AI PRO 300 Series Processors to Power Next Generation of Commercial PCs“.

You can also find out more information about the new 5th-gen EPYC processors here with the media release dubbed “AMD Launches 5th Gen AMD EPYC CPUs, Maintaining Leadership Performance and Features for the Modern Data Center.”

AMD’s MI-325X AI Accelerator processor details are here and in a media release titled “AMD Delivers Leadership AI Performance with AMD Instinct MI325X Accelerators.”

There’s also a wrap of the event titled “AMD Unveils Leadership AI Solutions at Advancing AI 2024“, which I’ve reprinted below, along with ALL the announcements and several NEW videos.

Here is a special media briefing on The Era of GenAI in AI PCs, presented by Adrian Macias, Sr. Director – Al Developer Acceleration, AMD Al Group – please watch – and there is PLENTY more below, as this is a comprehensive feature!

Do be sure to watch the October 2024 AMD Advancing AI keynote, embedded above, and please read on below!

Here are the other videos I recorded at the event, after which is the high level media release AMD shared on all of its announcements.

1. AMD has launched its latest high performance computing solutions defining the AI computing era, including 5th Gen AMD EPYC server CPUs, AMD Instinct MI325X accelerators, AMD Pensando Salina DPUs, AMD Pensando Pollara 400 NICs and AMD Ryzen AI PRO 300 series processors for enterprise AI PCs.

AMD and its partners also showcased how they are deploying AMD AI solutions at scale, the continued ecosystem growth of AMD ROCm open source AI software, and a broad portfolio of new solutions based on AMD Instinct accelerators, EPYC CPUs and Ryzen PRO CPUs.

“The data center and AI represent significant growth opportunities for AMD, and we are building strong momentum for our EPYC and AMD Instinct processors across a growing set of customers,” said AMD Chair and CEO Dr. Lisa Su. 

“With our new EPYC CPUs, AMD Instinct GPUs and Pensando DPUs we are delivering leadership compute to power our customers’ most important and demanding workloads. Looking ahead, we see the data center AI accelerator market growing to $500 billion by 2028. We are committed to delivering open innovation at scale through our expanded silicon, software, network and cluster-level solutions.”

Defining the Data Center in the AI Era

AMD announced a broad portfolio of data center solutions for AI, enterprise, cloud and mixed workloads:

New AMD EPYC 9005 Series processors deliver record-breaking performance to enable optimized compute solutions for diverse data center needs. Built on the latest “Zen 5” architecture, the lineup offers up to 192 cores and will be available in a wide range of platforms from leading OEMs and ODMs starting today.

AMD continues executing its annual cadence of AI accelerators with the launch of AMD Instinct MI325X, delivering leadership performance and memory capabilities for the most demanding AI workloads. AMD also shared new details on next-gen AMD Instinct MI350 series accelerators expected to launch in the second half of 2025, extending AMD Instinct leadership memory capacity and generative AI performance. AMD has made significant progress developing the AMD Instinct MI400 Series accelerators based on the AMD CDNA Next architecture, planned to be available in 2026.

AMD has continuously improved its AMD ROCm software stack, doubling AMD Instinct MI300X accelerator inferencing and training performance across a wide range of the most popular AI models. Today, over one million models run seamlessly out of the box on AMD Instinct, triple the number available when MI300X launched, with day-zero support for the most widely used models.

AMD also expanded its high performance networking portfolio to address evolving system networking requirements for AI infrastructure, maximizing CPU and GPU performance to deliver performance, scalability and efficiency across the entire system. The AMD Pensando Salina DPU delivers a high performance front-end network for AI systems, while the AMD Pensando Pollara 400, the first Ultra Ethernet Consortium ready NIC, reduces the complexity of performance tuning and helps improve time to production.

The article continues below, here’s more detail on AMD’s Pensando Networking achievements!

AMD partners detailed how they leverage AMD data center solutions to drive leadership generative AI capabilities, deliver cloud infrastructure used by millions of people daily and power on-prem and hybrid data centers for leading enterprises:

Since launching in December 2023, AMD Instinct MI300X accelerators have been deployed at scale by leading cloud, OEM and ODM partners and are serving millions of users daily on popular AI models, including OpenAI’s ChatGPT, Meta Llama and over one million open source models on the Hugging Face platform.

Google highlighted how AMD EPYC processors power a wide range of instances for AI, high performance, general purpose and confidential computing, including their AI Hypercomputer, a supercomputing architecture designed to maximize AI ROI. Google also announced EPYC 9005 Series-based VMs will be available in early 2025.

Oracle Cloud Infrastructure shared how it leverages AMD EPYC CPUs, AMD Instinct accelerators and Pensando DPUs to deliver fast, energy efficient compute and networking infrastructure for customers like Uber, Red Bull Powertrains, PayPal and Fireworks AI. OCI announced the new E6 compute platform powered by EPYC 9005 processors.

Databricks highlighted how its models and workflows run seamlessly on AMD Instinct and ROCm and disclosed that their testing shows the large memory capacity and compute capabilities of AMD Instinct MI300X GPUs help deliver an over 50% increase in performance on Llama and Databricks proprietary models.

Microsoft CEO Satya Nadella highlighted Microsoft’s longstanding collaboration and co-innovation with AMD across its product offerings and infrastructure, with MI300X delivering strong performance on Microsoft Azure and GPT workloads. Nadella and Su also discussed the companies’ deep partnership on the AMD Instinct roadmap and how Microsoft is planning to leverage future generations of AMD Instinct accelerators including MI350 series and beyond to deliver leadership performance-per-dollar-per-watt for AI applications.

Meta detailed how AMD EPYC CPUs and AMD Instinct accelerators power its compute infrastructure across AI deployments and services, with MI300X serving all live traffic on Llama 405B. Meta is also partnering with AMD to optimize AI performance from silicon, systems, and networking to software and applications.

Leading OEMs Dell, HPE, Lenovo and Supermicro are expanding on their highly performant, energy efficient AMD EPYC processor-based lineups with new platforms designed to modernize data centers for the AI era.

Expanding an Open AI Ecosystem

AMD continues to invest in the open AI ecosystem and expand the AMD ROCm open source software stack with new features, tools, optimizations and support to help developers extract the ultimate performance from AMD Instinct accelerators and deliver out-of-the-box support for today’s leading AI models. Leaders from Essential AI, Fireworks AI, Luma AI and Reka AI discussed how they are optimizing models across AMD hardware and software.

AMD also hosted a developer event joined by technical leaders from across the AI developer ecosystem, including Microsoft, OpenAI, Meta, Cohere, xAI and more. Luminary presentations hosted by the inventors of popular AI programming languages, models and frameworks critical to the AI transformation taking place, such as Triton, TensorFlow, vLLM and Paged Attention, FastChat and more, shared how developers are unlocking AI performance optimizations through vendor agnostic programming languages, accelerating models on AMD Instinct accelerators, and highlighted the ease of use porting to ROCm software and how the ecosystem is benefiting from an open-source approach.

Enabling Enterprise Productivity with AI PCs

AMD launched AMD Ryzen AI PRO 300 Series processors, powering the first Microsoft Copilot+ laptops enabled for the enterprise. The Ryzen AI PRO 300 Series processor lineup extends AMD leadership in performance and battery life with the addition of enterprise-grade security and manageability features for business users.

The Ryzen AI PRO 300 Series processors, featuring the new AMD “Zen 5” and AMD XDNA 2 architectures, are the world’s most advanced commercial processors, offering best in class performance for unmatched productivity and an industry leading 55 NPU TOPS of AI performance with the Ryzen AI 9 HX PRO 375 processor to process AI tasks locally on Ryzen AI PRO laptops.

Microsoft highlighted how Windows 11 Copilot+ and the Ryzen AI PRO 300 lineup are ready for next generation AI experiences, including new productivity and security features.

OEM partners including HP and Lenovo are expanding their commercial offerings with new PCs powered by Ryzen AI PRO 300 Series processors, with more than 100 platforms expected to come to market through 2025.

2. AMD Launches New Ryzen™ AI PRO 300 Series Processors to Power Next Generation of Commercial PCs

AMD has announced its third generation commercial AI mobile processors, designed specifically to transform business productivity with Copilot+ features including live captioning and language translation in conference calls and advanced AI image generators. The new Ryzen AI PRO 300 Series processors deliver industry-leading AI compute, with up to three times the AI performance than the previous generation, and offer uncompromising performance for everyday workloads. Enabled with AMD PRO Technologies, the Ryzen AI PRO 300 Series processors offer world-class security and manageability features designed to streamline IT operations and ensure exceptional ROI for businesses.

Ryzen AI PRO 300 Series processors feature new AMD “Zen 5” architecture, delivering outstanding CPU performance, and are the world’s best line up of commercial processors for Copilot+ enterprise PCs. Laptops equipped with Ryzen AI PRO 300 Series processors are designed to tackle business’ toughest workloads, with the top-of-stack Ryzen AI 9 HX PRO 375 offering up to 40% higher performance and up to 14% faster productivity performance compared to Intel’s Core Ultra 7 165U. With the addition of XDNA™ 2 architecture powering the integrated NPU, AMD Ryzen AI PRO 300 Series processors offer a cutting-edge 50+ NPU TOPS (Trillions of Operations Per Second) of AI processing power, exceeding Microsoft’s Copilot+ AI PC requirements, and delivering exceptional AI compute and productivity capabilities for the modern business. Built on a 4nm process and with innovative power management, the new processors deliver extended battery life ideal for sustained performance and productivity on the go.

“Enterprises are increasingly demanding more compute power and efficiency to drive their everyday tasks and most taxing workloads. We are excited to add the Ryzen AI PRO 300 Series, the most powerful AI processor built for business PCs, to our portfolio of mobile processors,” said Jack Huynh, senior vice president and general manager, Computing and Graphics Group at AMD. “Our third generation AI-enabled processors for business PCs deliver unprecedented AI processing capabilities with incredible battery life and seamless compatibility for the applications users depend on.”

AMD Ryzen AI Pro 300 specs

AMD Continues to Expand Commercial OEM Ecosystem

OEM partners continue to expand their commercial offerings with new PCs powered by Ryzen AI PRO 300 Series processors, delivering well-rounded performance and compatibility to their business customers. With industry leading TOPS, the next generation of Ryzen processor-powered commercial PCs are set to expand the possibilities of local AI processing with Microsoft Copilot+. OEM systems powered by Ryzen AI PRO 300 Series are expected to be on shelf starting later this year.

“Microsoft’s partnership with AMD and the integration of Ryzen AI PRO processors into Copilot+ PCs demonstrate our joint focus on delivering impactful AI-driven experiences for our customers. The Ryzen AI PRO’s performance, combined with the latest features in Windows 11, enhances productivity, efficiency, and security,” said Pavan Davuluri, corporate vice president, Windows+ Devices, Microsoft. “Features like Improved Windows Search, Recall, and Click to Do make PCs more intuitive and responsive. Security enhancements, including the Microsoft Pluton security processor and Windows Hello Enhanced Sign-in Security, help safeguard customer data with advanced protection. We’re proud of our strong history of collaboration with AMD and are thrilled to bring these innovations to market.”

“In today’s AI-powered era of computing, HP is dedicated to delivering powerful innovation and performance that revolutionises the way people work,” said Alex Cho, president of Personal Systems, HP. “With the HP EliteBook X Next-Gen AI PC, we are empowering modern leaders to push boundaries without compromising power or performance. We are proud to expand our AI PC lineup powered by AMD, providing our commercial customers with a truly personalised experience.”

“Lenovo’s partnership with AMD continues to drive AI PC innovation and deliver supreme performance for our business customers. Our recently announced ThinkPad T14s Gen 6 AMD, powered by the latest AMD Ryzen AI PRO 300 Series processors, showcases the strength of our collaboration,” said Luca Rossi, president, Lenovo Intelligent Devices Group. “This device offers outstanding AI computing power, enhanced security, and exceptional battery life, providing professionals with the tools they need to maximise productivity and efficiency. Together with AMD, we are transforming the business landscape by delivering smarter, AI-driven solutions that empower users to achieve more.”

New PRO Technologies Features Build Upon Leadership Security and Management Features

In addition to AMD Secure Processor, AMD Shadow Stack and AMD Platform Secure Boot, AMD has expanded its PRO Technologies lineup with new security and manageability features. Processors equipped with PRO Technologies will now come standard with Cloud Bare Metal Recovery, allowing IT teams to seamlessly recover systems via the cloud ensuring smooth and continuous operations; Supply Chain Security (AMD Device Identity), a new supply chain security function, enabling traceability across the supply chain; and Watch Dog Timer, building on existing resiliency support with additional detection and recovery processes.

Additional AI-based malware detection is available via PRO Technologies with select ISV partners. These new security features leverage the integrated NPU to run AI-based security workloads without impacting day-to-day performance.

3. AMD Launches 5th Gen AMD EPYC CPUs, Maintaining Leadership Performance and Features for the Modern Data Center

AMD has announced the availability of the 5th Gen AMD EPYC™ processors, formerly codenamed “Turin,” the world’s best server CPU for enterprise, AI and cloud.

Using the “Zen 5” core architecture, compatible with the broadly deployed SP5 platform and offering a broad range of core counts spanning from 8 to 192, the AMD EPYC 9005 Series processors extend the record-breaking performance and energy efficiency of the previous generations with the top of stack 192 core CPU delivering up to 2.7X the performance compared to the competition.

New to the AMD EPYC 9005 Series CPUs is the 64 core AMD EPYC 9575F, tailor made for GPU powered AI solutions that need the ultimate in host CPU capabilities. Boosting up to 5GHz, compared to the 3.8GHz processor of the competition, it provides up to 28% faster processing needed to keep GPUs fed with data for demanding AI workloads.

“From powering the world’s fastest supercomputers, to leading enterprises, to the largest Hyperscalers, AMD has earned the trust of customers who value demonstrated performance, innovation and energy efficiency,” said Dan McNamara, senior vice president and general manager, server business, AMD. “With five generations of on-time roadmap execution, AMD has proven it can meet the needs of the data center market and give customers the standard for data centre performance, efficiency, solutions and capabilities for cloud, enterprise and AI workloads.”

The World’s Best CPU for Enterprise, AI and Cloud Workloads

Modern data centers run a variety of workloads, from supporting corporate AI-enablement initiatives, to powering large-scale cloud-based infrastructures to hosting the most demanding business-critical applications. The new 5th Gen AMD EPYC processors provide leading performance and capabilities for the broad spectrum of server workloads driving business IT today.

The new “Zen 5” core architecture, provides up to 17% better instructions per clock (IPC) for enterprise and cloud workloads and up to 37% higher IPC in AI and high performance computing (HPC) compared to “Zen 4.”

With AMD EPYC 9965 processor-based servers, customers can expect significant impact in their real world applications and workloads compared to the Intel Xeon® 8592+ CPU-based servers, with:

  • Up to 4X faster time to results on business applications such as video transcoding.
  • Up to 3.9X the time to insights for science and HPC applications that solve the world’s most challenging problems.[8]
  • Up to 1.6X the performance per core in virtualized infrastructure.

In addition to leadership performance and efficiency in general purpose workloads, 5th Gen AMD EPYC processors enable customers to drive fast time to insights and deployments for AI deployments, whether they are running a CPU or a CPU + GPU solution.

Compared to the competition:

  • The 192 core EPYC 9965 CPU has up to 3.7X the performance on end-to-end AI workloads, like TPCx-AI (derivative), which are critical for driving an efficient approach to generative AI.
  • In small and medium size enterprise-class generative AI models, like Meta’s Llama 3.1-8B, the EPYC 9965 provides 1.9X the throughput performance compared to the competition.
  • Finally, the purpose built AI host node CPU, the EPYC 9575F, can use its 5GHz max frequency boost to help a 1,000 node AI cluster drive up to 700,000 more inference tokens per second. Accomplishing more, faster.

By modernising to a data centre powered by these new processors to achieve 391,000 units of SPECrate®2017_int_base general purpose computing performance, customers receive impressive performance for various workloads, while gaining the ability to use an estimated 71% less power and ~87% fewer servers. This gives CIOs the flexibility to either benefit from the space and power savings or add performance for day-to-day IT tasks while delivering impressive AI performance.

AMD EPYC CPUs – Driving Next Wave of Innovation

The proven performance and deep ecosystem support across partners and customers have driven widespread adoption of EPYC CPUs to power the most demanding computing tasks. With leading performance, features and density, AMD EPYC CPUs help customers drive value in their data centres and IT environments quickly and efficiently.

5th Gen AMD EPYC Features

The entire lineup of 5th Gen AMD EPYC processors is available today, with support from Cisco, Dell, Hewlett Packard Enterprise, Lenovo and Supermicro as well as all major ODMs and cloud service providers providing a simple upgrade path for organisations seeking compute and AI leadership.

High level features of the AMD EPYC 9005 series CPUs include:

  • Leadership core count options from 8 to 192, per CPU
  • “Zen 5” and “Zen 5c” core architectures
  • 12 channels of DDR5 memory per CPU
  • Support for up to DDR5-6400 MT/s
  • Leadership boost frequencies up to 5GHz5
  • AVX-512 with the full 512b data path
  • Trusted I/O for Confidential Computing, and FIPS certification in process for every part in the series

Here is the big Media Q&A from the event;

Here is the smaller EPYC 5th-gen Turin Media Q&A:

4. AMD Delivers Leadership AI Performance with AMD Instinct MI325X Accelerators

AMD has announced the latest accelerator and networking solutions that will power the next generation of AI infrastructure at scale: AMD Instinct™ MI325X accelerators, the AMD Pensando™ Pollara 400 NIC and the AMD Pensando Salina DPU. AMD Instinct MI325X accelerators set a new standard in performance for Gen AI models and data centers.

Built on the AMD CDNA™ 3 architecture, AMD Instinct MI325X accelerators are designed for exceptional performance and efficiency for demanding AI tasks spanning foundation model training, fine-tuning and inferencing. Together, these products enable AMD customers and partners to create highly performant and optimized AI solutions at the system, rack and data center level.

“AMD continues to deliver on our roadmap, offering customers the performance they need and the choice they want, to bring AI infrastructure, at scale, to market faster,” said Forrest Norrod, executive vice president and general manager, Data Center Solutions Business Group, AMD. “With the new AMD Instinct accelerators, EPYC processors and AMD Pensando networking engines, the continued growth of our open software ecosystem, and the ability to tie this all together into optimised AI infrastructure, AMD underscores the critical expertise to build and deploy world class AI solutions.”

AMD Instinct MI325X Extends Leading AI Performance

AMD Instinct MI325X accelerators deliver industry-leading memory capacity and bandwidth, with 256GB of HBM3E supporting 6.0TB/s offering 1.8X more capacity and 1.3x more bandwidth than the H2001. The AMD Instinct MI325X also offers 1.3X greater peak theoretical FP16 and FP8 compute performance compared to H200

This leadership memory and compute can provide up to 1.3X the inference performance on Mistral 7B at FP16, 1.2X the inference performance on Llama 3.1 70B at FP8 and 1.4X the inference performance on Mixtral 8x7B at FP16 of the H200.

AMD Instinct MI325X accelerators are currently on track for production shipments in Q4 2024 and are expected to have widespread system availability from a broad set of platform providers, including Dell Technologies, Eviden, Gigabyte, Hewlett Packard Enterprise, Lenovo, Supermicro and others starting in Q1 2025.

Continuing its commitment to an annual roadmap cadence, AMD previewed the next-generation AMD Instinct MI350 series accelerators. Based on the AMD CDNA 4 architecture, AMD Instinct MI350 series accelerators are designed to deliver a 35x improvement in inference performance compared to AMD CDNA 3-based accelerators.

The AMD Instinct MI350 series will continue to drive memory capacity leadership with up to 288GB of HBM3E memory per accelerator. The AMD Instinct MI350 series accelerators are on track to be available during the second half of 2025.

AMD Next-Gen AI Networking

AMD is leveraging the most widely deployed programmable DPU for hyperscalers to power next-gen AI networking. Split into two parts: the front-end, which delivers data and information to an AI cluster, and the backend, which manages data transfer between accelerators and clusters, AI networking is critical to ensuring CPUs and accelerators are utilized efficiently in AI infrastructure.

To effectively manage these two networks and drive high performance, scalability and efficiency across the entire system, AMD introduced the AMD Pensando™ Salina DPU for the front-end and the AMD Pensando™ Pollara 400, the industry’s first Ultra Ethernet Consortium (UEC) ready AI NIC, for the back-end.

The AMD Pensando Salina DPU is the third generation of the world’s most performant and programmable DPU, bringing up to 2X the performance, bandwidth and scale compared to the previous generation. Supporting 400G throughput for fast data transfer rates, the AMD Pensando Salina DPU is a critical component in AI front-end network clusters, optimizing performance, efficiency, security and scalability for data-driven AI applications.

The UEC-ready AMD Pensando Pollara 400, powered by the AMD P4 Programmable engine, is the industry’s first UEC-ready AI NIC. It supports the next-gen RDMA software and is backed by an open ecosystem of networking. The AMD Pensando Pollara 400 is critical for providing leadership performance, scalability and efficiency of accelerator-to-accelerator communication in back-end networks.

Both the AMD Pensando Salina DPU and AMD Pensando Pollara 400 are sampling with customers in Q4’24 and are on track for availability in the first half of 2025.

AMD AI Software Delivering New Capabilities for Generative AI

AMD continues its investment in driving software capabilities and the open ecosystem to deliver powerful new features and capabilities in the AMD ROCm™ open software stack.

Within the open software community, AMD is driving support for AMD compute engines in the most widely used AI frameworks, libraries and models including PyTorch, Triton, Hugging Face and many others. This work translates to out-of-the-box performance and support with AMD Instinct accelerators on popular generative AI models like Stable Diffusion 3, Meta Llama 3, 3.1 and 3.2 and more than one million models at Hugging Face.

Beyond the community, AMD continues to advance its ROCm open software stack, bringing the latest features to support leading training and inference on Generative AI workloads. ROCm 6.2 now includes support for critical AI features like FP8 datatype, Flash Attention 3, Kernel Fusion and more. With these new additions, ROCm 6.2, compared to ROCm 6.0, provides up to a 2.4X performance improvement on inference and 1.8X on training for a variety of LLMs.

Here is the smaller media Q&A for the AMD Instinct that I attended:

Here is the session where Dr Lisa Su closes the event for the media and announces the WINNERS of AMD’s tech competition:

Here is AMD’s video on AMD and AI helping solve the most important challenges: