AI Servers Can Reach “Rack Density” Of 1000+ KW Likely With NVIDIA’s Next-Gen “Rubin Ultra” Architecture

It is now being predicted that individual server racks could reach a potential rack density of over 1,000 kilowatts with next-gen architectures such as NVIDIA’s Rubin Ultra chips, highlighting the rise in energy consumption by the AI sector.

AI Servers Are Squeezing In Massive Energy As The World Looks Towards Acquiring More AI Compute Power Onboard With Next-Gen Chips From NVIDIA & AMD

While the world is in dire need of having more AI computing power onboard, it is essential to know that the “greed of power” comes with several “unacknowledged” downsides, notably the rapid increase in energy consumption.

Modern-day AI clusters are reported to have a rack density of around 130 KW-250 KW, and according to an analysis by the data center firm Vertiv (via BusinessWire), this number is expected to soar over 1,000 KW, which is a significant milestone along with concern for global leaders on how to cater to the massive energy demand.

For those unaware, rack density is a term used to define the amount of heat generated and power required to run all the devices in a network or server rack. Interestingly, back in 2020, the average rack density was rated to be around 8.2 KW only. With the rapid rise in AI demand, this number has soared tremendously in just four years. This not only shows how quickly a market segment can innovate if it sees mainstream attention, but the gigantic rise in server density also depicts that we are on the right track when it comes to having more computing power.

1000 kilowatt AI server racks incoming by end of decade pic.twitter.com/zzzI1UcMS5

— tae kim (@firstadopter) November 19, 2024

Speaking of the 1,000 KW figure, Vertiv’s analysis says that this server rack density benchmark will be obtained with the debut of NVIDIA’s “Rubin Ultra” AI servers. This shows that the next-gen lineup will pack a serious performance punch and is likely to categorize itself as “revolutionary” for the markets.

While the high server rack density figures are indeed something to appreciate, it is important to note that this will ultimately translate into the need to produce more energy. A while ago, we reported that NVIDIA’s global data centers are consuming power equivalent to that of individual nations, and given that this figure surfaced months ago, the actual number right now will be far larger, especially with the debut of next-gen AI products.

The industry has resorted to solutions to cater to the growing energy needs, and one way is to use nuclear-power infrastructure for data centers. It was rumored that Microsoft and OpenAI are working on something similar, although we are not quite certain yet, but given that the power consumption figures continue to soar like their current trend, we will definitely see the markets opting for such alternatives.