Name: How the Chips That Power AI Work
Uploaded: 2023-12-19T05:01:00Z
Duration: 6 min 29 s
Description: The technology behind generative AI like ChatGPT has exploded, fueling a demand for chips that can handle the processing power these programs need. WSJ visited Amazon’s chip lab to see how these chips work, and why tech titans think they are the future. Illustration: John McColgan

https://www.barrons.com/client

Topics

Memberships

Tools

Customer Service

Network

Newsstand

For Business

For Education

For Advertisers

Account

About Barron's

bolded

unBolded

Ken Burns, the filmmaker behind the documentary miniseries "The American Revolution" and others, spoke to Barron's editor at large Andy Serwer.

https://m.wsj.net/video/20260507/0291a39f-ea0f-4e3b-8f5f-c6783863e213/1/hls/manifest-hd-wifi.m3u8

Filmmaker Ken Burns on 'the Most Important Revolution in History'

Jane Fraser, CEO of Citigroup, discusses how they scale their business, their global footprint, and investing in digital assets.

https://m.wsj.net/video/20260506/c85686e1-e814-47e7-a3dc-72a0170fa2c9/1/hls/manifest-hd-wifi.m3u8

Citigroup CEO on Keeping Competitors at Bay

As CEO of Citigroup, Jane Fraser tells Barron’s Andy Serwer about her goals for the future.

https://m.wsj.net/video/20260506/904afc41-63ad-4d30-94d5-fe292999af50/1/hls/manifest-hd-wifi.m3u8

Planning for Growth at Citigroup

Jane Fraser, CEO of Citigroup, sat down for an interview with Barron's Editor at Large Andy Serwer.

https://m.wsj.net/video/20260430/9cbc994b-3c3c-4ab2-994c-1363ccf00ade/1/hls/manifest-hd-wifi.m3u8

Citigroup CEO: 'We're Cheap. Get on the Train.'

Michel Khalaf, chief executive officer of MetLife, discusses how he’s navigating the current market.

https://m.wsj.net/video/20260501/91689664-d41c-4215-8400-ea340e71cf8d/1/hls/manifest-hd-wifi.m3u8

Risk Management in a K-Shaped Economy

“I’m a big believer in a level playing field,” Michel Khalaf, chief executive officer of MetLife, tells Barron’s Andy Serwer.

https://m.wsj.net/video/20260501/0a4674b6-10e7-4a91-bd96-c6a2e6aa7fe2/1/hls/manifest-hd-wifi.m3u8

Why MetLife’s CEO Supports Private Equity in Insurance

Michel Khalaf, chief executive officer of MetLife, spoke to Barron's editor at large Andy Serwer.

https://m.wsj.net/video/20260423/d5471223-ead4-4616-a52a-63b6722abe1e/1/hls/manifest-hd-wifi.m3u8

MetLife's CEO Explains How AI Has Helped the Insurance Company

Constantine Alexandrakis, CEO of the executive search firm Russell Reynolds Associates, discusses the pattern of higher turnover rates and shorter tenures of CEOs.

https://m.wsj.net/video/20260501/2fec28b4-35fb-4ff1-9864-55d5538b0973/1/hls/manifest-hd-wifi.m3u8

Being a CEO Is Becoming More Complicated, According to This Executive

At Barron's

"Your 9-5...is just not going to do it," Nick Magguilli, personal finance expert, said on Barron's Streetwise podcast.

https://m.wsj.net/video/20251216/e6897805-d131-4cc4-9c59-ca03b0c0b958/1/hls/manifest-hd-wifi.m3u8

How to Scale Your Net Worth

TJ Thornton, Head of Global Research Product Marketing at BofA Securities, talks to Jack Hough about his favorite stocks outside of AI.

https://m.wsj.net/video/20251210/f9c1a95c-5a0e-4349-9fe4-73b3ac53bfb8/1/hls/manifest-hd-wifi.m3u8

BofA’s Favorite Non-AI Stocks

Jack Hough talks to TJ Thornton, Head of Global Research Product Marketing at BofA Securities about diversifying stocks to hedge AI risks.

https://m.wsj.net/video/20251210/8e164ff7-1500-4c2f-8713-e891c640a0f4/1/hls/manifest-hd-wifi.m3u8

Evaluating Investing Opportunities Beyond AI

Steve Quirk, chief brokerage officer at Robinhood, breaks down the value of tokenizing stocks in different markets.

https://m.wsj.net/video/20251126/f86d65ea-bd8b-4777-b6b0-80f7bb12027e/1/hls/manifest-hd-wifi.m3u8

Unpacking Tokenization for Everyday Investors

Jack Hough talks to Steve Quirk, chief brokerage officer at Robinhood, about trends in retail investing.

https://m.wsj.net/video/20251126/a9cbdcc6-f060-4fc9-a15e-396924e25946/1/hls/manifest-hd-wifi.m3u8

How Robinhood Reshaped the Trading Landscape for an Entire Generation

Streetwise host Jack Hough sat down with strategist Scott Wren to discuss how the AI-fueled market run stacks up against the dot-com boom.

https://m.wsj.net/video/20251111/bd2e1a5b-1a6e-4543-8133-dcc63f69b5dc/1/hls/manifest-hd-wifi.m3u8

AI vs. The Dot Com Bubble

Bank of America’s Haim Israel shares what’s next for Nuclear Energy development.

https://m.wsj.net/video/20250930/ab4fc336-af44-4af2-8e40-9ae6f04a445e/1/hls/manifest-hd-wifi.m3u8

The Outlook For Nuclear

Jack talks to Lyft CEO David Risher about growth potential in the company.

https://m.wsj.net/video/20250929/ae62b70e-912b-4ce7-ae63-1b2652628bae/1/hls/manifest-hd-wifi.m3u8

Lyft's Ride to Profit

The stock market is expensive. What does that mean for your portfolio? Jack sits down with Franklin Templeton’s Matt Quinlan to talk dividend investing, and whether it’s a good move in a pricey market.

https://m.wsj.net/video/20250911/9c85e132-d710-4458-9467-aa69d4f25ae0/1/hls/manifest-hd-wifi.m3u8

Dividends and Capital Appreciation

Barron's Streetwise

CEO Chris Todd explains how the company is providing core technology to RIAs and broker dealers while building infrastructure to meet future advisor needs in areas like alternative investments and digital currencies.

https://m.wsj.net/video/20260512/06d41382-0347-4765-b253-994badc172a3/1/hls/manifest-hd-wifi.m3u8

Envestnet’s ‘Bullseye’ Strategic Approach  

Interviews with top wealth management professionals about topics including compensation, hiring, marketing, next-gen, and more.

CEO Chris Todd says the company is using AI to increase internal productivity and leveraging its large data set to provide "investment data insights" to help advisors manage client portfolios.

https://m.wsj.net/video/20260512/4207fb3b-4df7-43b2-8c1e-32d3cb75967f/1/hls/manifest-hd-wifi.m3u8

How Envestnet Is Harnessing AI and Data 

Yie-Hsin Hung, president and CEO of State Street Investment Management, describes how the company addresses retirement challenges like increased longevity and the shift of responsibility from pensions to individuals.

https://m.wsj.net/video/20260428/266b9598-bb99-4025-9b2e-d17b76104723/1/hls/manifest-hd-wifi.m3u8

Empowering Investors to Better Prepare for Retirement

Yie-Hsin Hung, president and CEO of State Street Investment Management, says that retirement is a vital business segment, accounting for about a quarter of the company’s global assets.

https://m.wsj.net/video/20260428/ea1e71f0-04bc-4b92-9b82-e52911542bd5/1/hls/manifest-hd-wifi.m3u8

Retirement Is a ‘Very Important Part’ of State Street’s Business

ETF flows reveal a “widening out” of exposure away from large caps, says Yie-Hsin Hung, president and CEO of State Street Investment Management.

https://m.wsj.net/video/20260428/735ae074-7b28-4394-879a-5959b7675c9e/1/hls/manifest-hd-wifi.m3u8

U.S. Investors Are Embracing Small-Cap and Mid-Cap Stocks

Yie-Hsin Hung, president and CEO, says the company is focused on growth in fixed income products, international markets, and the overall wealth category.

https://m.wsj.net/video/20260428/718c379d-03d5-4861-be33-fa3c4f56cef7/1/hls/manifest-hd-wifi.m3u8

Where State Street Investment Management Sees Opportunity

Within five years, “the consumer will have the ability to go online and design their financial advisor—what they look like, how they talk, how they interact, what they remember,” says Rob Nelson, CEO and founder of NorthRock Partners.

https://m.wsj.net/video/20260415/09483094-28db-4623-a9b2-df5b9327dffa/1/hls/manifest-hd-wifi.m3u8

The Future AI Advisor

Rob Nelson, CEO and founder of NorthRock Partners, explains why he is optimistic about the future of financial advice and the potential for advisors to form meaningful client communities.

https://m.wsj.net/video/20260415/4bc66985-8aab-4e8a-b646-8e3023cdd9b0/1/hls/manifest-hd-wifi.m3u8

The Jevons Paradox and Placing a Premium on Community

Rob Nelson, CEO and founder of NorthRock Partners, says clients today place a premium on having free time and peace of mind.

https://m.wsj.net/video/20260415/528964a5-1d56-41bf-9931-bf152533679b/1/hls/manifest-hd-wifi.m3u8

How Client Expectations Have Changed in the Past 15 Years

Barron's Advisor The Way Forward

A “willingness to roll up your sleeves and understand the issues that your clients are facing, not just from an investment perspective,” is common among the industry’s most successful, says Kelly Gaur, a Mercer Advisors regional vice president.

https://m.wsj.net/video/20260319/318adb2f-a888-4f30-b182-c791e27f98ab/1/hls/manifest-hd-wifi.m3u8

Best Advice for Someone Considering Wealth Management as a Career

Kelly Gaur, a Mercer Advisors regional vice president, says she was drawn to working for a registered investment advisor partly because they generally offer a more personalized, modern approach to wealth management.

https://m.wsj.net/video/20260318/5575e9ca-0b48-4fb9-8e43-82aae1b3e3fc/1/hls/manifest-hd-wifi.m3u8

The Appeal of Working for an RIA

“The best client development professionals are those who have served clients in some capacity,” says Kelly Gaur, a Mercer Advisors regional vice president.

https://m.wsj.net/video/20260318/c1638929-6b21-4573-9deb-4f78e1ffa1d7/1/hls/manifest-hd-wifi.m3u8

Client-Facing Experience Can Help Advance Your Career

The Mercer Advisors regional vice president discusses her business development role and tips for breaking into the industry and gaining valuable experience in a non-advisory capacity.

https://m.wsj.net/video/20260318/f2c2c9a5-cb67-4167-a48b-7fe4a9285615/1/hls/manifest-hd-wifi.m3u8

Kelly Gaur: Tips for Pursuing Non-Advisor Roles in Wealth Management

“You want to make sure that everybody's excited and prepared and that you're all pulling the rope in the same direction,” says Nate Fisher, managing director and wealth advisor at Carson Wealth.

https://m.wsj.net/video/20260305/20285ef5-c115-435f-9291-e4f2d5b166a9/1/hls/manifest-hd-wifi.m3u8

A Successful Transition Requires Honesty and a ‘Heavy Lift’

Young advisors considering a succession plan should maximize engagement with clients because “it can take a long time to gain that trust and build those relationships,” says Nate Fisher, managing director and wealth advisor at Carson Wealth.

https://m.wsj.net/video/20260305/236fdaa2-5acb-48e9-b16d-e20b58846487/1/hls/manifest-hd-wifi.m3u8

Succession Planning and Establishing Trust With Clients

The Carson Wealth managing director and wealth advisor says his decision to partner with the firm was driven by the need for increased capacity for holistic services and a clear succession plan for his retiring father.

https://m.wsj.net/video/20260305/33bb1217-c8be-4358-ba17-6238cbab6fa1/1/hls/manifest-hd-wifi.m3u8

Nate Fisher: Lessons Learned From a Successful Partnership

As trust grows, advisors tend to spend less time discussing investments and more time on clients’ families and well-being, says Nate Fisher, managing director and wealth advisor at Carson Wealth.

https://m.wsj.net/video/20260305/d2d3903a-994f-4ae7-82be-706929930b3d/1/hls/manifest-hd-wifi.m3u8

Trusted Advisors Focus Less on Investments, More on Clients' Lives 

Serving tech founders is a full-time, long-term effort that often requires significant upfront work before a successful, revenue-generating relationship is established, says Katie Partridge, a Morgan Stanley private wealth advisor.

https://m.wsj.net/video/20260219/abc63bad-4c7a-474b-b4a8-b4feae461375/1/hls/manifest-hd-wifi.m3u8

Landing Founders as Clients Requires Patience and Hard Work

Barron's Advisor Next Generation

Government policy shapes retirement regardless of savings or lifespan. With Social Security's trust fund facing a 2033 shortfall, what fixes are coming? Our experts discuss policy shifts and how to budget your benefits.

https://m.wsj.net/video/20260410/aea80278-298b-458e-8bcd-f378796d6328/2/hls/manifest-hd-wifi.m3u8

What's Next for Social Security?

Stocks swoop and soar as investors weigh AI hype against rate-cut "tea leaves." Our panelists explain how to drown out the noise, chart your own winning path,and identify investments that offer true long-term rewards.

https://m.wsj.net/video/20260410/586aecf7-117b-4f3e-86f0-6869d907e671/2/hls/manifest-hd-wifi.m3u8

Riding the Market Roller Coaster

The capture of Venezuela’s Nicolás Maduro and Middle East tensions are reshaping oil supply expectations. The U.S. is intervening in energy markets in decades-unseen ways. Experts break down what’s next for oil, gas and energy stocks.

https://m.wsj.net/video/20260212/9779ec13-602a-4a38-bb73-495ff464a24f/1/hls/manifest-hd-wifi.m3u8

Oil and Gas in a Volatile World

For the first time in decades, U.S. power demand is rising - driven by AI data centers, EVs and electrification. Utility and clean-energy stocks are climbing, but costly buildouts threaten affordability. We explore what’s next for the power trade.

https://m.wsj.net/video/20260212/81a81fdd-47b4-44f6-93d1-7597a7fab032/1/hls/manifest-hd-wifi.m3u8

The Gold Rush in Electricity

Will Nvidia and other technology companies keep powering the stock market in 2026? We'll speak with two veteran money managers about their views on tech and their favorite stocks for the coming year.

https://m.wsj.net/video/20251212/5683d7a0-ed1f-4e3c-b913-bb3367ea3abd/1/hls/manifest-hd-wifi.m3u8

Finding Bargains in a Pricey Market

Explore what could go right for investors, and what could go wrong, in a year that will see the AI trade evolve, the Fed's leadership change, and global competition for resources increase in this 2026 Outlook.

https://m.wsj.net/video/20251212/573f5954-c4ab-43e9-804d-f1ea93fbb70c/1/hls/manifest-hd-wifi.m3u8

Meeting the Challenge

Stocks are expensive, risks abound, and AI is about to change the world. Other than that, building a calamity-proof retirement portfolio should be a cinch. Tune in as two veteran money managers size up the markets and share their investment ideas.

https://m.wsj.net/video/20250211/17a5e7d6-7c36-44b0-a35e-bf5ac3a24194/1/hls/manifest-hd-wifi.m3u8

Saving for a Sunny Day

Research shows that people greatly underestimate the healthcare costs they’ll face in retirement. Learn from experts how to estimate those costs, prepare for potential long-term care needs, and manage healthcare expenses in retirement.

https://m.wsj.net/video/20250211/e7c022d9-badf-4e95-8e7c-0b6fce9f7823/1/hls/manifest-hd-wifi.m3u8

Are You Fiscally Fit?

Surging power demand, new energy sources, and growing competition have spurred the utility industry to become more innovative than ever before. Our panelists will unpack the changes afoot and explore what lies ahead for companies, customers, and investors.

https://m.wsj.net/video/20240917/001ea098-1fcc-4da5-9f0e-a3b149bbb2fb/1/hls/manifest-hd-wifi.m3u8

Not Your Father's Utility

Barron's Roundtable

Whether you're leading a transition, building a scalable team, or just trying to stay ahead of the curve, this episode will give you practical insights to lead smarter and grow stronger.

https://m.wsj.net/video/20251010/fd26d60b-d1e6-409b-b51b-101da4aaa368/1/hls/manifest-hd-wifi.m3u8

Next-Gen Leadership Meets Smart Innovation

Electric vehicles sales are growing but not at the pace some expected. Some would-be buyers worry about the range of EV charging, and others are swayed by politics. But innovators from Detroit to Silicon Valley still envision a dramatic shift in how we get around. What’s going on and what’s next?

https://m.wsj.net/video/20240603/5784fa60-791c-47ad-9740-42e4553de5c4/1/hls/manifest-hd-wifi.m3u8

Forces Shaping EV Adoption and the Future of Driving

Success in improving the power grid to accommodate the electricity demands of the future will be a boon for companies doing the work and utilities that succeed. Explore the investing risks and opportunities.

https://m.wsj.net/video/20240522/2b825c69-2687-4d88-923b-0c8f523a2f67/1/hls/manifest-hd-wifi.m3u8

How to Invest in Initiatives to Bolster the Power Grid

Joanna Rotenberg, Fidelity's president of personal investing, discusses ways to make investing more accessible and how to bolster your own financial security.

https://m.wsj.net/video/20231025/8ad733df-ac90-41fd-9655-7761b3e0532c/2/hls/manifest-hd-wifi.m3u8

Mind the Gap: Demystifying Investing

Reshma Kapadia sits down with Cozy Wittman of College Inside Track to discuss how to tackle all things college, from what admission officers are seeking today, to navigating the different ways to funding higher education and evaluating the ROIs.

https://m.wsj.net/video/20230515/b7229011-05cc-42ec-9c16-1d7752ec668d/1/hls/manifest-hd-wifi.m3u8

Decoding the College Search and Financing Puzzle

Nicole Webb, financial advisor and SVP at Wealth Enhancement Group, shares ways to navigate the real estate market this spring whether you are a first-time buyer, a retiree downsizing or someone who is relocating or looking for a second home.

https://m.wsj.net/video/20230424/93eb00e1-caf0-4e83-9fc7-b502a57dc546/1/hls/manifest-hd-wifi.m3u8

Pro Tips on Buying and Selling a Home

Reggie Fils-Aimé, former president and COO of Nintendo of America, joins Barron's deputy editor Alex Eule to discuss his unlikely rise to the top, and the risks he took once there to beat the odds and achieve success.

https://m.wsj.net/video/20220630/61e2296d-ab1a-4216-ae65-24676447c616/1/hls/manifest-hd-wifi.m3u8

How Nintendo's Former President Disrupted the Gaming Industry

2050 Wealth Planners co-CEO Lazetta Rainey Braxton, Sequoia Financial Planners vice president Elizabeth Scheiderer, and MoneyZen founder Manisha Thakor offer tips on how to prepare for recession, inflation, caregiving needs.

https://m.wsj.net/video/20220617/a71025ce-518b-49b9-b098-21417863f8ce/1/hls/manifest-hd-wifi.m3u8

Financial Advice for the Inflation-Weary

Families and advisors have a lot to navigate when it comes to generational wealth transfer. Add sustainable investing to the list. Examine strategies to help close the generational divide. From the Dow Jones Energy & Sustainability event series.

https://m.wsj.net/video/20230210/85cbf3f9-b5ab-4bfa-93c6-f8e8a247d386/1/hls/manifest-hd-wifi.m3u8

Sustainable Investing and the Wealth Transfer

Barron's Events

Level Up

Barron's Explains

Back to Business

Great Investors

MarketWatch

Video Center - Barron's home

The technology behind generative AI like ChatGPT has exploded, fueling a demand for chips that can handle the processing power these programs need. WSJ visited Amazon’s chip lab to see how these chips work, and why tech titans think they are the future. Illustration: John McColgan

https://m.wsj.net/video/20231219/af42fa22-4304-492d-8a1c-49ba1626c700/1/hls/manifest-hd-wifi.m3u8

How the Chips That Power AI Work

Narrator: Over the last year, everyone has been talking about.

Generative AI.

Generative AI.

Generative AI.

Generative AI.

I'm Like, wait, why am I doing this? I just wait for the AI to do it.

Narrator: Driving the boom are AI chips. Some no bigger than the size of your palm, and the demand for them has skyrocketed.

We originally thought the total market for data center, AI accelerators would be about 150 billion, and now we think it's gonna be over 400 billion.

Narrator: As AI gains popularity, some of the world's tech titans are racing to design chips that run better and faster. Here's how they work and why tech companies are betting they're the future. This is "The Tech Behind AI Chips." This is Amazon's chip lab in Austin, Texas, where the company designs AI chips to use in AWS's servers.

Right out of manufacturing, we get something that is called the wafer.

Narrator: Ron Diamant is the chief architect of Inferentia and Trainium, the company's custom AI chips.

These are the compute elements or the components that actually perform the computation.

Each of these rectangles, called dice, is a chip. Each die contains tens of billions of microscopic semiconductors called transistors that communicate inputs and outputs.

Think about one millionth of a centimeter, that's roughly the size of each one of these transistors.

Narrator: All chips use semiconductors like this. What makes AI chips different from CPUs, the kind of chip that powers your computer or phone, is how they're packaged. Say, for example, you want to generate a new image of a cat. CPUs have a smaller number of powerful cores. The units that make up the chip that are good at doing a lot of different things, these cores process information sequentially. So one calculation after another. So to create a brand new image of a cat, it would only produce a couple pixels at a time. But an AI chip has more cores that run in parallel, so it can process hundreds or even thousands of those cat pixels all at once. These cores are smaller and typically do less than CPU cores but are specially designed for running AI calculations. But those chips can't operate on their own.

That compute die then gets integrated into a package, and that's what people typically think about when they think about the chip.

Amazon makes two different AI chips, named for its two essential functions, training and inference. Training is where an AI model is set millions of examples of something, images of cats, for instance, to teach it what a cat is and what it looks like. Inference is when it uses that training to actually generate an original image of a cat. Training is the most difficult part of this process.

We typically train not on one chip but rather on tens of thousands of chips. In contrast, inference is typically done on 1 to 16 chips.

Narrator: Processing all of that information demands a lot of energy, which generates heat.

And we're able to use this device here in order to force a certain temperature to the chip, and that's how we're able to test that the chip is reliable at very low temperatures and very high temperatures.

Narrator: To help keep chips cool, they're attached to heat sinks, pieces of metal with vents that help dissipate heat. Once they're packaged, the chips are integrated into servers for Amazon's AWS cloud.

So the training cards will be mounted on this baseboard, eight of them in total, and they are interconnected between them at a very high bandwidth and low latency. So this allows the different training devices inside the server to work together on the same training job. So if you are interacting with an AI chatbot, your text, your question will hit the CPUs, and the CPUs will move the data into the Inferentia2 devices, which will collectively perform a gigantic computation. Basically performing the AI model, will respond to the CPU with the result, and the CPU will send the result back to you.

Narrator: Amazon's chips are just one type competing in this emerging market, which is currently dominated by the biggest chip designer, Nvidia.

Nvidia is still a chip provider to all different types of customers who have to run different workloads. And then the next category of competitor that you have is the major cloud providers. Microsoft, Amazon, AWS, and Google are all designing their own chips because they can optimize their computing workloads for the software that runs on their cloud to get a performance edge, and they don't have to give Nvidia its very juicy profit margin on the sale of every chip.

Narrator: But right now, generative AI is still a young technology. It's mostly used in consumer-facing products like chatbots and image generators, but experts say that hype cycles around technology can pay off in the end.

While there might be something like a dot-com bubble for the current AI hype cycle, at the end of the dot-com bubble was still the internet. And I think we're in a similar situation with generative AI.

Narrator: The technology's rapid advance means that chips and the software to use them are going to have to keep up. Amazon says it uses a mixture of its own chips and Nvidia's chips to give customers multiple options. Microsoft says it's following a similar model.

For those cloud providers, the question is, how much of their computing workloads for AI is gonna be offered through Nvidia versus their own custom AI chips? And that's the battle that's playing out in corporate boardrooms all over the world.

Narrator: Amazon released a new version of Trainium in November. Diamant says he doesn't see the AI boom slowing down anytime soon.

We've been investing in machine learning and artificial intelligence for almost two decades now, and we're just seeing a step-up in pace of innovation and capabilities that these models are enabling us. So our investment in AI chips is here to stay with a significant step-up in capabilities from generation to generation.