We’re Building the Future of Data Infrastructure

June 12, 2024

How AI Will Change the Building Blocks of Semis

By Michael Kanellos, Head of Influencer Relations, Marvell

Aaron Thean, points to a slide featuring the downtown skylines of New York, Singapore and San Francisco along with a prototype of a 3D processor and asks, “Which one of these things is not like the other?”

The answer? While most gravitate to the processor, San Francisco is a better answer. With a population well under 1 million, the city’s internal transportation and communications systems don’t come close to the level of complexity, performance and synchronization required by the other three.

With future chips, “we’re talking about trillions of transistors on multiple substrates,” said Thean, the deputy president of the National University of Singapore and the director of SHINE, an initiative to expand Singapore’s role in the development of chipets, during a one-day summit sponsored by Marvell and the university.

Advanced packaging, new semiconductor materials, and innovative types of interconnects have long been on the horizon for chip developers as technologies that can continue to deliver the steady drumbeat of power and performance gains once achieved through Moore’s Law.

AI has changed the picture. The surge in demand for AI services, and the growing sophistication demanded by consumers and businesses, are accelerating chip roadmaps.

“Semiconductors are the foundation of AI,” said Dr. Loi Nguyen, executive vice president and general manager of cloud optics at Marvell. “Without them, there would be no AI.”

The catch? Designing devices from a palette of disparate subcomponents that are economical, sustainable and reliable—yet deliver outsized performance gains—will be far from easy. Future accelerators will consume tens of kilowatts and might contain twelve or more discrete processing units along with massive amounts of HBM. In this environment, small changes in design can mean magnified impacts on heat dissipation and latency, among other factors.

“Power will be the most challenging aspect of AI,” said Alfred Yeo, director of research and development at StatsChipPAC, a subsidiary of the JCET group specializing in semiconductor integration and packaging services.

Optical, Copper, Lithium and Glass

So what’s next?

A larger role for optics, which consumes less power and delivers greater performance than traditional copper electrical connections, is almost a certainty.

“Over the next three to five years, we will need to start to bring optical onto the chip,” said Nguyen.

Silicon photonics or SiPho, the technology for producing the hundreds of devices required for optical communications with comparatively inexpensive silicon processes, will also play a role. SiPho is already used extensively in coherent optics for connecting data centers. They will appear next in the modules connecting systems like servers and switching within a datacenter. After that will come electro-optic modulators and other devices inside for connecting cores to each other, larger internal caches of high-bandwidth memory, and/or non-volatile storage inside the chiplet package.

More optical technology will also help reduce the number of times signals must be converted between optical and electrical devices. “Each one of those transitions is super costly in terms of power and latency,” said Noam Mizrahi, executive vice president and corporate CTO at Marvell.

Still, copper and electrical circuits will roll on. Metal bumps will remain the medium of choice for chip stacking and TSVs (through silicon vias). Copper is more economical for the short chip-to-chip links.

“At short distances, electrons are great. At longer distances, electrons suck,” joked Thean. “Over millimeter-scale distances, that is where optical is great.”

Packaging will also experience a burst of innovation. Designers will experiment with fan-out and fan-up designs, different types of bonding, vertical stacking and other techniques as they seek optimal combinations of performance, power dissipation, volumetric size and efficiency. The formulas for different types of devices will differ. The aspect that won’t change is the fact that future devices won’t be monolithic; they will be made of subcomponents.

“If you want to go past certain bottlenecks, you have to integrate,” said Radha Nagarajan, Marvell senior vice president and CTO of optical platforms.

Likewise, materials such as lithium niobate and silicon carbide will be used to develop subcomponents in chiplets. In some situations, glass substrates could replace silicon.

“Silicon will be around forever, but silicon won’t do everything that we need,” added Thean.

A Changing Industry

The emphasis on heterogenous device design and chiplets will also likely have an impact on the structure of the industry because device design will require a wide range of disciplines. It will be expected that entrepreneurs launch specialized packaging firms, fabs and prototyping services. The shift toward heterogenous development could play into Singapore’s strengths in manufacturing and project management, noted Beh Kian Teik, CEO of the National Science Foundation, which is helping cultivate an ecosystem that’s bringing university R&D in closer contact with established companies.

“We all have a sense that the impact of AI will be pervasive and profound,” Teik said. “But we need more people in the sector, and we will need more powerful, efficient hardware.”

# # #

Marvell and the M logo are trademarks of Marvell or its affiliates. Please visit www.marvell.com for a complete list of Marvell trademarks. Other names and brands may be claimed as the property of others.

This blog contains forward-looking statements within the meaning of the federal securities laws that involve risks and uncertainties. Forward-looking statements include, without limitation, any statement that may predict, forecast, indicate or imply future events, results or achievements. Actual events, results or achievements may differ materially from those contemplated in this blog. Forward-looking statements are only predictions and are subject to risks, uncertainties and assumptions that are difficult to predict, including those described in the “Risk Factors” section of our Annual Reports on Form 10-K, Quarterly Reports on Form 10-Q and other documents filed by us from time to time with the SEC. Forward-looking statements speak only as of the date they are made. Readers are cautioned not to put undue reliance on forward-looking statements, and no person assumes any obligation to update or revise any such forward-looking statements, whether as a result of new information, future events or otherwise.

Tags: AI

Recent Posts

Archives

Categories

Accelerated Infrastructure is the Foundation of AI