To address the multi-trillion-dollar opportunity promised by generative AI, cloud hyperscalers are investing hundreds of billions into AI infrastructure. This makes the AI cloud the latest and fastest-growing addition to a data-center universe that includes on-premises, high-performance computing, and general-purpose cloud computing.
Scaling the AI cloud involves significant challenges across multiple dimensions including scaling up compute, scaling out compute, and minimizing total cost of ownership (TCO).
AI scaling requirements are so demanding that a new class of data-center infrastructure is needed to address them: accelerated infrastructure. With its leadership position in custom accelerated infrastructure silicon, Marvell is at the forefront of this new realm.
Whether accelerated or general-purpose, the Marvell data-center portfolio addresses every product category, from AI accelerators to custom CPUs to high-speed copper and optical interconnects to high-bandwidth network switches to storage and memory devices.
Accelerated Infrastructure
Accelerated Infrastructure
Accelerated Infrastructure
AI is orders of magnitude more data-hungry than any previous application. AI workloads must be distributed across multiple accelerated compute devices; and these AI accelerators must function as a single computing element.
Accelerated infrastructure refers to the collection of silicon and associated hardware needed make this possible in hyperscale AI data centers.
- Compute encompasses any accelerated computing device used in the AI data center, including custom XPUs, CPUs and DPUs.
- Switching refers to network switches used in frontend and backend networks as well as compute fabrics. These AI switching domains must be of far higher capacity than those in general-purpose computing and must exhibit predictable latency.
- Interconnect refers to the vast array of electrical and optical devices used to provide high-bandwidth connectivity inside of and between AI servers, and between racks, rows, buildings and campuses.
With a broad and deep accelerated infrastructure portfolio and two decades of custom silicon development experience, Marvell is uniquely capable of delivering platform solutions tailored to the needs of each hyperscaler.
Automotive Compute
Compute
Compute
Accelerated compute is at the heart of the AI revolution. But raw compute performance is not the sole consideration. Hyperscalers are moving quickly to optimize their AI infrastructure, including accelerated compute, for their unique requirements. The imperative to optimize is driven by three factors:
- Scale. The sheer scale of AI deployments amplifies gains associated with small improvements and costs associated with minor inefficiencies.
- Differentiation. To remain competitive, hyperscalers must differentiate their service offerings. Optimized hardware and use of their own intellectual property helps to achieve this end.
- Unique use cases. Each hyperscaler pursues different business models, including internal applications, SaaS, and IaaS; and offers unique applications within each. The compute requirements for each application vary such that merchant silicon may not be most efficient or optimal.
To optimize their infrastructure, hyperscalers are turning to Marvell to co-develop and customize compute devices optimized for their application-specific performance and total cost of ownership (TCO) objectives. Marvell brings a 20-year history of custom expertise to the design of leading-edge devices such as AI accelerators, CPUs, NICs, storage and memory.
Scale Up
Scale Up
Scale Up
Every hyperscaler faces the challenge of scaling up AI compute—enabling tens or hundreds or thousands of AI accelerators (XPUs) to effectively act as a single supercomputer. Doing so requires a scale-up fabric to connect these XPUs that’s characterized by extreme bandwidth, ultra-low latency, and lossless, deterministic performance.
Scaling up AI compute is not a simple matter of choosing the best technology. Rather, it’s a question of the best tool for the job.
- One approach may be best for training, another best for inference.
- A scale-up architecture designed to support the workload of a single user will be different than one supporting multi-tenancy.
- The needs of a 100-XPU scale-up domain are different than those of a 1000-XPU domain.
Ultimately, each AI scale-up fabric must be optimized for the specific workloads to be processed. Marvell brings to this challenge high-bandwidth copper, optical and silicon photonics expertise, switching fabric know-how and a legacy of leading-edge custom silicon development.
Scale Out
Scale Out
Scale Out
In the AI cloud, scaling out is just as important to breaking AI performance barriers as is scaling up. Scale out involves adding more nodes to the infrastructure, increasing capacity by distributing workloads across tens of thousands or even millions of geographically dispersed accelerators.
Scaling out presents its own optimization challenges:
- Different cluster sizes yield differing requirements for attributes such as port speed, radix and transmission medium across racks and rows.
- Different deployment environments, such as a building versus a campus, yield differing priorities for interconnect attributes such as power, latency and fiber capacity.
- Scale-out networks for phased deployments within a data center require switch fabric attributes, such as open standards, distinct from those of one-time, data-center-scale deployments.
Marvell offers a comprehensive scale-out portfolio addressing every interconnect distance, power, and latency regime, enabling AI architects to select the optimum interconnect technology for each deployment scenario. These are complemented by Marvell Ethernet network switch devices that minimize network latency while providing open-source deployment flexibility.
Discover Our Data Center Portfolio
Resources
Marvell Announces Breakthrough Co-Packaged Optics Architecture for Custom AI Accelerators
Learn More
Custom Compute in the AI Era
Learn More
Custom, Copper and Cross-Country Connectivity: Eight Big Trends for Marvell in 2024
Learn More
