Layer 3: Infrastructure

The third layer is where chips become a working system. Huang lists its ingredients plainly: land, power delivery, cooling and networking, the physical systems that let tens of thousands of processors work as one. He gives the result a name that has stuck: the AI factory.

Warehouse vs. factory

The name is deliberate. A traditional data center is closer to a warehouse. It stores information (your photos, emails and web pages) and hands it back when asked. The same request gets the same file.

An AI factory makes something. Power and prompts go in, and new tokens come out: answers, code, images, decisions that didn't exist a moment ago. In Huang's framing, these buildings are designed to manufacture intelligence, not to store it.

The four ingredients

• Land and buildings. Big AI sites cover the area of several football fields, and they need to be where power, water and fiber connections can reach them.
• Power delivery. Getting electricity from the grid to each rack takes substations, transformers and backup systems. A single rack of modern AI servers can draw as much power as dozens of homes.
• Cooling. All that power becomes heat, packed into a small space. Air cooling is no longer enough for the densest racks, so many new sites pipe liquid right up to the chips.
• Networking. This is the least obvious ingredient and one of the most important. Training a large model spreads the work across thousands of chips that constantly swap results. If the network is slow, every chip waits. Fast networking is what lets a building full of separate machines act like one computer.

Why this layer takes years

Chips can be ordered. An AI factory has to be built: sites found, permits won, grid connections approved, and buildings, power systems and cooling plants constructed and wired by skilled workers. That's why Huang stresses that the AI buildout creates demand for electricians, plumbers, construction workers, steelworkers and network technicians, not just software engineers.

It's also where most of the money goes. Huang says a few hundred billion dollars has been spent on this buildout so far, and that trillions more will be needed. Whether that forecast holds is one of the big open questions in AI, and we'll come back to it in the last chapter.

That's the physical half of the cake: power, chips and the factories that combine them. Next, the layers most people actually picture when they hear "AI."