Digitally generated image of 3d cubes connected, hovering above a computer or server motherboard.Digitally generated image of 3d cubes connected, hovering above a computer or server motherboard.

PowerEdge

Dell Generative AI Open Ecosystem with AMD Instinct Accelerators

The world’s broadest GenAI solutions portfolio just got better.
By   |  

Generative AI (GenAI) is the decade’s most promising accelerator for innovation with 78% of IT decision makers reporting they’re largely excited for the potential GenAI can have on their organizations.¹ Most see GenAI as a means to provide productivity gains, streamline processes and achieve cost savings. Harnessing this technology is critical to ensure organizations can compete in this new digital era.

Dell Technologies and AMD are coming together to unveil an expansion to the Dell Generative AI Solutions portfolio, continuing the work of accelerating advanced workloads and offering businesses more choice to continue their unique GenAI journeys. This new technology highlights a pivotal role played by open ecosystems and silicon diversity in empowering customers with simple, trusted and tailored solutions to bring AI to their data.

The surging growth of data and the question of what to make of it are central in today’s AI landscape. AI is the latest wave of innovation, and working to help ensure organizations have access to the compute power needed to fuel their innovation engine is key. Together with AMD, we can offer customers more choice to power their AI and GenAI workloads than before.

The fastest ramping solution in Dell history,² the Dell PowerEdge XE9680, will support the upcoming AMD Instinct™ MI300X accelerator, offering customers more choice when it comes to AI performance. The PowerEdge XE9680 with AMD Instinct MI300X offers high-performance capabilities for enterprises seeking to unlock the value of their data and differentiate their business with customized large language models (LLMs). With eight MI300X GPUs, 192GB of 5.3 TB/s High Bandwidth Memory (HBM3) per GPU for a total coherent HBM3 capacity of 1.5 TB per server and over 21 petaflops of FP16 performance, the PowerEdge XE9680 with MI300X is poised to further democratize access to GenAI for enterprises eager to train larger models, reduce data center footprints, lower TCO and gain a competitive edge.

Along with the support for the AMD MI300X Instinct Accelerator, we’re also announcing the next step of Dell Generative AI Solutions, making it easier for organizations to deploy trustworthy GenAI with the new Dell Validated Design for Generative AI with AMD.

This new Dell Validated Design for Generative AI with AMD ROCm™ powered AI frameworks, available next year, extends the Dell Generative AI Solutions ecosystem and will include open-source large language models. AMD ROCm™ is an open-source stack of drivers, dev toolkits and APIs for AMD Instinct accelerators. Dell Validated Designs for Generative AI make it simple for Dell customers to build trusted GenAI platforms tailored to their needs by taking the guesswork out of integration, performance and sizing considerations.

Dell’s latest eBook and infographic demonstrate how ROCm-optimized open-source AI and ML frameworks like PyTorch, TensorFlow, ONNX-RT, JAX and OpenAI Triton have native support for the PowerEdge XE9680 with MI300X, enabling simple drop-in support for LLMs based on these popular frameworks. Read our three-part blog series to learn more and explore code samples:

Part 1: Is AMD ROCm ready to deploy leading AI workloads?

Part 2: How to run Hugging Face models with AMD ROCm on Dell PowerEdge

Part 3: How to run Llama 2 via Hugging Face models with AMD ROCm

The AMD Instinct™ MI300X accelerator, which will be available for the Dell PowerEdge XE9680 server.
8x AMD Instinct MI300X accelerator Open Compute Project Accelerator Module (OAM) Universal Baseboard (UBB).

The PowerEdge XE9680 is a powerhouse in performance and a champion of security and simplicity. Dell Open Manage Enterprise software simplifies and accelerates GenAI infrastructure deployment, while CloudIQ software provides intelligent automation from anywhere. The PowerEdge XE9680 is backed by integrated cyber recovery and a broad Zero Trust approach for greater security.

The PowerEdge XE9680 leverages the AMD Instinct Platform powered by eight AMD Instinct MI300X accelerators, enabling near-linear scaling and low latency distributed GenAI training and inferencing with Global Memory Interconnect (xGMI) spanning a cluster of PowerEdge servers interconnecting MI300X GPUs over an Ethernet-based AI fabric using a Dell validated GPU fabric with Dell PowerSwitch Z9664F-ON.

As a member of the Ultra Ethernet Consortium (UEC), Dell continues to advocate for standards-based networking. Dell Technologies is uniquely positioned to realize the full potential of an open approach by implementing it across the entire GenAI solution: compute, fabric and storage. The Dell PowerEdge XE9680 with AMD Instinct™ MI300X accelerators and Dell Validated Design for Generative AI with AMD will be available in the first half of 2024.

Learn more about the Dell PowerEdge XE family of accelerated servers.

See the latest on Dell’s growing range of acceleration products for AI, including the AMD Instinct MI300X.

AMD, the AMD Arrow logo, AMD Instinct, and combinations thereof are trademarks of Advanced Micro Devices.
1 Dell Technologies Generative AI Pulse Survey, October 2023
2 Based on Dell analysis, August 2023

About the Author: Dave Lincoln

Dave Lincoln is Vice President of Compute Systems, including responsibility for the Dell PowerEdge server line of business supporting the enterprise, cloud, AI, HPC, edge, and telco segments. Compute platforms are one of the engines that help drive our clients’ growth and success. Dave joined Dell’s Infrastructure Group in March 2017 leading Server Sales Strategy and Business Operations before moving to Client Group in April 2018 to lead CPG Strategy. Shortly afterwards, he was selected to serve as the Vice President of Dell’s Fixed Computing Business Unit, comprised of the OptiPlex desktop and Wyse thin client businesses. Dave returned to the Infrastructure Group in February 2021 to lead Dell’s Networking and Emerging Server lines of business. Prior to Dell, Dave served in leadership roles across multiple technology companies including most recently at Lenovo where he led the Storage, Converged and Software Defined Business Units. Originally from Detroit, Dave graduated from Michigan State University with a Supply Chain degree and later received his MBA from Duke University. He and his family live in North Carolina’s Research Triangle area.