TECH NEWS – Jensen Huang, CEO of Nvidia, made a somewhat odd comment during a Q&A session after his Computex presentation.

The power consumption of a stack of AI-focused GPUs can be monstrously high, and Huang, who these days lives by the motto “the more you buy, the more you save” with these chips, has similar thoughts about the power consumption associated with the latest AI models, as they mostly run on Nvidia hardware. Nvidia’s GPUs are faster and more efficient than their rivals, and he believes that speeding everything up will save time and energy.

“Generative AI is not about training, it’s about inference. The goal is not training, the goal is inference. When you do inference, the amount of energy you use is much, much less than the alternative way of doing computation. For example, I showed you the climate simulation in Taiwan – 3,000 times less energy. Not 30% less, 3,000 times less. This is happening in application after application. AI doesn’t care where it goes to school.

The world doesn’t have enough power close to the population. But the world has a lot of excess energy. Just the amount of energy that comes from the sun is incredible. But it’s in the wrong place. We should put power plants and data centers for training where we don’t have population, train the model somewhere else. Move the model for inference everyone’s closer to the people – in their pocket on phones, PCs, data centers and cloud data centers,” Huang said.

The World Economic Forum (WEF) recently estimated that AI-related energy consumption will grow by 26-36% per year and could reach the level of a small country (e.g. Iceland) by 2028. But reducing consumption would require significant efficiency improvements, because the faster the task is performed, the more energy is required.

Nvidia’s AI GPU starts at around $40,000, so it’s a pretty high bar to clear for an AI startup to invest in such hardware.

Source: PCGamer