❄️ Liquid Cooling vs. Air Cooling for High-Density AI Racks

Why AI Racks Broke the Old Thermal Math

For twenty years, data center thermal design followed a predictable trajectory: rack power density crept up slowly, and air cooling kept pace. A typical enterprise server rack running virtualized workloads or web infrastructure draws somewhere between 5 and 15 kW. Hot aisle/cold aisle containment, raised-floor air distribution, and CRAC/CRAH units sized to match were a mature, well-understood engineering problem. Facility engineers could design to that envelope with confidence.

GPU-based AI training and inference infrastructure abandoned that envelope almost overnight. A single NVIDIA H100 GPU has a thermal design power (TDP) of roughly 700 W. An 8-GPU server built around that generation of accelerator can draw 10 kW or more by itself — before adding CPUs, memory, NVMe storage, and networking. Pack a handful of those servers into a single rack and density routinely reaches 40 to 100+ kW per rack, with the newest B200-class and rack-scale systems (NVIDIA GB200 NVL72 and similar) pushing well past 100 kW in a single cabinet. That is a five-to-twenty-fold increase in heat load over a conventional enterprise rack occupying the identical floor footprint.

This is not a scaling problem that more fans and colder air can solve. Air has a fixed, comparatively low heat capacity, and moving enough of it to reject 100 kW from a single rack requires airflow volumes and fan power that become physically and economically absurd well before you get there. Liquid — water or a dielectric fluid — has roughly 3,000 times the volumetric heat capacity of air, which is why nearly every hyperscaler and enterprise deploying GPU clusters at scale has moved to some form of liquid cooling. The question facing most facility engineers today isn’t whether liquid cooling is coming, but which approach to adopt, and how to get there from an existing air-cooled facility.

Rack Power Density: Three Tiers

• Traditional enterprise IT (5–15 kW/rack): General-purpose virtualization, web/app servers, storage arrays. Standard hot/cold aisle containment with CRAC/CRAH units handles this comfortably at typical 1.4–1.6 PUE.
• High-density air-cooled (15–30 kW/rack): Dense compute or early GPU deployments pushed to the edge of what raised-floor air cooling can deliver. Requires tightly engineered containment, higher static pressure, in-row cooling units, and careful CFD-validated airflow — workable, but with shrinking margin and rising fan energy.
• Liquid-cooled AI racks (40–100+ kW/rack): Modern multi-GPU training and inference clusters. Air alone cannot reject this heat load in a standard rack footprint; some form of liquid cooling becomes structurally necessary, not optional.

Air Cooling Fundamentals and Its Practical Ceiling

Air cooling remains the default for a reason: it is mature, well understood, and requires no plumbing inside the white space. The standard architecture pairs hot aisle/cold aisle containment with CRAC (Computer Room Air Conditioning, DX-based) or CRAH (Computer Room Air Handler, chilled-water-based) units that push conditioned air through a raised floor plenum and perforated tiles into the cold aisle, where it is drawn through the servers and exhausted into a contained hot aisle for return.

This architecture has a hard physical ceiling, and it is governed by simple thermodynamics: the heat carried away by air is proportional to airflow volume (CFM) times the temperature rise across the equipment. As rack power climbs, you either need dramatically more airflow or a much larger allowable temperature rise — and both options run out of headroom quickly.

• Airflow volume scales unfavorably. Rejecting 30 kW from a single rack with a reasonable 20–25°F (11–14°C) delta-T requires roughly 4,000–5,000 CFM through that one rack alone. At 80–100 kW, the required airflow becomes difficult to deliver through standard floor tiles and rack intakes without excessive velocity, turbulence, and recirculation.
• Fan power rises with the cube of airflow. Fan power is proportional to the cube of volumetric flow rate for a fixed system, so doubling airflow to handle higher density can increase fan energy roughly eightfold. Past a certain point, the energy spent moving air outweighs the practicality of the approach and directly erodes PUE.
• Raised-floor plenum depth and tile open area become limiting factors. Most legacy floors were engineered for 150–300 CFM per tile. Getting several times that volume through the same floor cutout without starving neighboring racks requires deeper plenums, high-flow grates, or supplemental in-row/in-rack fan units — retrofits that are disruptive and only extend the ceiling modestly.
• CRAC/CRAH capacity and redundancy math gets expensive fast. Concentrating 60–80 kW into one rack while neighboring racks sit at 8 kW creates a hot spot the room-level cooling plant wasn’t designed around, forcing either localized supplemental cooling or significant over-provisioning of room-level capacity that sits idle everywhere else.

In practice, most experienced data center engineers treat roughly 25–30 kW per rack as the point where air cooling shifts from “efficient and comfortable” to “technically possible but increasingly expensive and operationally fragile.” Beyond that, some form of liquid cooling stops being a forward-looking option and becomes the pragmatic choice.

Liquid Cooling Approaches

1. Direct-to-Chip (Cold Plate) Cooling

Direct-to-chip — also called cold-plate cooling — routes a liquid coolant loop directly to a metal cold plate mounted on top of the CPU and GPU packages, replacing the heat sink and fan assembly on the highest-power components. The coolant (typically a water/glycol mixture in a closed loop, isolated from facility water by a CDU) absorbs heat at the chip and carries it out of the rack to a coolant distribution unit, which rejects it to the facility chilled water loop or an outdoor dry cooler.

Critically, direct-to-chip only captures the highest-density heat sources — typically 60–80% of a GPU server’s total heat load. Memory, VRMs, NICs, storage, and power supplies remain air-cooled, so the rack still needs some air handling, just at a much-reduced level. This makes direct-to-chip the most common “first step” into liquid cooling: it slots into a standard rack footprint, works with mostly conventional server chassis (with a liquid-cooled variant of the board), and can often be retrofitted into an existing air-cooled data hall by adding CDUs and piping without changing the room-level architecture.

2. Rear-Door Heat Exchangers (RDHx)

A rear-door heat exchanger replaces the standard perforated rear door of an otherwise conventional air-cooled rack with a liquid-cooled coil. Servers inside the rack still draw cold aisle air through themselves with their own internal fans exactly as before — nothing changes inside the server. The difference is what happens on exit: instead of dumping hot exhaust air into the hot aisle, the air passes through the liquid-cooled door coil, which removes the heat before the air re-enters the room. Done well, RDHx can return air to the room at or near cold-aisle temperature, effectively neutralizing the rack’s contribution to the hot aisle.

RDHx is attractive as a retrofit because it requires no changes to the servers themselves — any standard rack and any standard server can sit behind an RDHx door. It is typically deployed to push existing air-cooled racks from the 15–30 kW range up toward 35–45 kW without touching room-level CRAC/CRAH capacity, since the heat is captured and removed at the rack rather than dumped into the room. It does not, however, reduce fan energy inside the servers, and it still depends on facility chilled water or a dedicated liquid loop reaching each rack.

3. Immersion Cooling

Immersion cooling submerges entire servers — boards, GPUs, memory, drives, and all — directly in a dielectric (electrically non-conductive) fluid that does not damage the electronics. It comes in two forms:

• Single-phase immersion: Servers sit in an open tank of mineral oil or a synthetic dielectric fluid that stays liquid throughout the cycle; the warmed fluid is pumped out, cooled through a heat exchanger, and recirculated.
• Two-phase immersion: Uses an engineered fluid with a low boiling point that vaporizes on contact with hot components, carries heat away as it changes phase, then condenses on a cooled coil at the top of a sealed tank and drips back down — a passive, highly efficient phase-change cycle.

Because the entire server is bathed in fluid, immersion eliminates server fans and most room-level air-handling equipment entirely — there is effectively no meaningful air-side heat load left to manage. This makes immersion the highest-density option, capable of supporting the upper end of the AI rack range and beyond in a compact tank footprint, and it typically delivers the best PUE of any approach. The tradeoff is the most significant departure from conventional data center design: tanks replace rack rows, servers require fluid-compatible components and modified maintenance procedures (no hot-swapping bare boards in a running tank the way you would in air), and floor loading, fluid handling, and fire protection all need re-engineering. Immersion is generally best suited to greenfield builds or dedicated high-density zones rather than incremental retrofits of existing air-cooled halls.

Tradeoff Comparison

Facility-Level Implications

Every liquid cooling approach introduces infrastructure a pure air-cooled facility doesn’t have: a coolant distribution unit (CDU) to isolate the technology cooling loop from facility water and control supply temperature and flow, piping and manifolds routed to the rack or row level, leak detection, and — for direct-to-chip and RDHx — quick-disconnect fittings engineered to minimize spill risk during service. None of this is exotic anymore; CDUs are now a standard catalog item from major thermal management vendors, but it is capital and commissioning scope that a traditional raised-floor design never had to account for.

The payoff is twofold. First, density: liquid cooling lets a facility pack several times the compute into the same floor footprint, which matters enormously as GPU TDPs continue climbing — an 8-GPU server that draws 10+ kW today will be exceeded by the next accelerator generation, and air-cooled halls simply run out of runway. Second, PUE: removing server and room-level fan energy from the equation, and in many designs enabling higher facility water temperatures that unlock more hours of free cooling or eliminate mechanical chilling entirely, is one of the few remaining levers that meaningfully moves PUE once a facility has already optimized its air-side design. Hyperscale operators reporting industry-leading PUE figures (often below 1.1) are almost universally running some form of liquid cooling for their highest-density compute.

How to Approach the Decision

There is no single correct answer — the right approach depends on where you’re starting from and what you’re trying to achieve:

• Existing air-cooled facility, moderate GPU deployment: Rear-door heat exchangers are usually the fastest, lowest-disruption path to push existing racks from the 15–30 kW range into the 35–45 kW range without touching room-level cooling plant capacity or server hardware.
• Existing facility, larger or growing GPU cluster: Direct-to-chip is the most common choice once density exceeds what RDHx alone can handle. It requires liquid-capable server SKUs and CDU infrastructure but integrates into a conventional rack row layout and data hall.
• Greenfield build or a dedicated high-density zone at very large scale: Immersion cooling delivers the highest density and best PUE, and is easiest to justify when you’re not constrained by an existing raised floor, rack standard, or service procedure — new construction can be engineered around tanks from the start.
• Budget and PUE/sustainability targets: If a facility is contractually or regulatorily bound to an aggressive PUE target, or if energy cost is the dominant driver, the OpEx savings from liquid cooling — particularly immersion — typically justify the higher upfront CapEx within a few years at any meaningful GPU cluster scale.
• Hybrid strategies are common and often correct. Many operators run direct-to-chip for GPU-dense racks alongside conventional air cooling for lower-density storage and networking racks in the same hall, rather than converting an entire facility to a single cooling technology.

The through-line for any facility engineer evaluating this decision: identify the actual per-rack density your GPU roadmap requires over the next two to three procurement cycles, not just the density of today’s deployment, and select the cooling architecture with headroom to match — retrofitting a second time is far more expensive than sizing correctly the first time.