TECH NEWS – Forcing CUDA technology onto gaming graphics cards triggered something close to an existential crisis for Nvidia. It is almost surreal in hindsight, because CUDA is now one of the company’s greatest strengths, but according to Jensen Huang, the decision nearly broke Nvidia before it turned into the moat that defines the company today.
It is astonishing to see how indispensable CUDA has become for Nvidia and its broader business, especially considering that, according to Jensen Huang, the framework created in 2006 is now one of the main reasons the company holds such a dominant position in AI. Yet, as Huang explained on Lex Fridman’s podcast, CUDA represented a major risk when it was introduced, because Nvidia wanted to be seen not merely as a GPU maker but as a full-scale computing platform provider. Huang did not want the company boxed into a narrow specialization. He wanted Nvidia to expand across the entire computing landscape.
Huang described how the idea behind CUDA, in other words programmable GPUs, emerged and how programmable pixel shaders helped make that possible by allowing GPUs to be used for more than just 3D graphics workloads. With pixel shaders, Nvidia imagined a world in which programmability could move beyond the limits of general-purpose CPUs and enter a new domain. According to Huang, the arrival of pixel shaders marked Nvidia’s true entry into computing, but even then coding on the GPU was not as precise as what other tools could achieve. He said the next huge breakthrough was support for FP32 calculations in programmable shaders, which finally allowed Nvidia to push into a market where researchers and experts wanted to use GPUs for compute-heavy tasks.
The most fascinating part of the CUDA story is that once Nvidia had achieved GPU programmability, it still had to keep investing in and expanding the idea. At the time, however, Nvidia’s customer base was focused primarily on rendering games, so the company was not expecting programmable GPUs to generate immediate revenue. Huang and his team knew that putting CUDA directly onto GeForce GPUs would not produce short-term financial returns, which made it an expensive gamble that increased costs significantly while gross margins were shrinking. It took roughly a decade for the CUDA bet to pay off, and Huang’s patience, along with that of the development team in maintaining the software stack without seeing practical results for years, is one of the reasons CUDA eventually developed into the ecosystem it is today. Huang remains grateful to GeForce for being the engine that pushed CUDA and its software ecosystem forward.
“The better we become at computing, the worse we become at specializing. The more we specialize, the less capacity we have for overall computing. The company has to find that really narrow path, step by step, to expand our computing capabilities without giving up our most important specialization. That’s why people working on stream processors and other kinds of dataflow processors found us. It cost us enormous amounts of profit, and at the time we could not afford it. But we did it anyway because we wanted to be a computing company. We increased our costs by 50% while we were a company operating at a 35% gross margin. Our market cap dropped to about one and a half billion dollars. You could imagine that one day this would go into workstations and supercomputers, and maybe we could capture higher margins there. So you could reason your way into why we could afford to do it. I always say Nvidia is the house that GeForce built – because GeForce is what brought CUDA to everyone.” – Huang said.
And today, the number of CUDA cores on an Nvidia GPU really does matter a lot…
Source: WCCFTech
