THE CRUNCH
Google has released TorchTPU, its version of the popular PyTorch AI toolkit, as an open-source project. It lets AI apps run on Google's Tensor Processing Units (TPUs) and makes it easy to move existing PyTorch apps to TPUs and Google Cloud with a simple configuration change and no rewriting. With the move, Google joins Amazon in trying to break Nvidia's hold on AI computing, where most AI apps are written by default to run on Nvidia's GPUs.
PyTorch is the open-source toolkit used, by some accounts, to build 85% of AI apps. Meta created it in 2016, when Nvidia's GPUs were already the standard hardware for AI research, so many of its features were built specifically for those chips. Meta handed the project to the Linux Foundation in 2022.
The timing is pointed: Amazon made a similar announcement on Monday, releasing TorchNeuron to port AI apps to its homegrown Trainium chip and AWS, also with no rewriting. Amazon is going further, submitting code to the main PyTorch project that decouples features from Nvidia's chips so the toolkit is more open to any hardware by default.
Google may be one-upping its rival in places. It says it worked directly with Meta, Hugging Face, Inferact (maker of vLLM) and RadixArk (maker of SGLang) to ensure TorchTPU works well with popular AI development tools, and it has launched a TPU agent suite to help developers move apps to its chips and fix errors along the way. It also updated LiteRT, its tool for running AI on phones, laptops and embedded devices, to work better with PyTorch apps.
None of this means developers will ditch Nvidia's hardware en masse just because alternatives are now easier to reach, but both cloud rivals are clearly working to end Nvidia's advantage.