Amazon is taking its biggest shot at Nvidia yet with new TorchNeuron dev tool

TorchNeuron is a tool adapted from PyTorch, the open-source toolkit developers use to build, train, and run AI models. Trainium is Amazon’s AI chip. TorchNeuron helps applications written in PyTorch run on Trainium, connecting those applications to Amazon Web Services. That matters because startups can use Amazon’s cloud infrastructure while choosing hardware beyond Nvidia’s GPUs.

The main mechanism is simple configuration. Amazon says developers can move a PyTorch application to Trainium by changing one line, without rewriting the application’s code. AWS also says Trainium can handle both training and inference, so one chip supports the full model workload. Deepgram reportedly moved to Trainium using native PyTorch support and processes 70% more audio per dollar.

TorchNeuron is part of Amazon’s broader effort to make PyTorch less tied to Nvidia hardware. AWS argues that startups gain not only a chip option, but also cloud tools for moving from prototypes to production. The company says this can improve overall economics, even if AWS does not offer the lowest per-token price.

PyTorch plays a central role in AI development. Amazon says about 85% of AI applications are written with it. The toolkit is free and open source, and developers use it to build, train, and run models. Since 2022, PyTorch has been part of the Linux Foundation, which maintains open-source projects.

This large developer base makes compatibility highly valuable to chip companies. If an application already uses PyTorch, a hardware maker can reduce the work needed to move it onto another chip. Amazon’s TorchNeuron is designed around that advantage. It lets developers use their existing PyTorch knowledge and, Amazon says, move applications to Trainium with one configuration change.

Nvidia has benefited from PyTorch support for years through CUDA, its software infrastructure for running applications on Nvidia hardware. That support helped make Nvidia hardware deeply embedded in the toolkit. Competing chip companies therefore want PyTorch to work smoothly with their products. Amazon is also contributing changes intended to make PyTorch more hardware-agnostic.

TorchNeuron works as an adaptation of PyTorch rather than requiring developers to learn an entirely different software system. Because developers already know PyTorch, Amazon says they can use that knowledge when moving an application to Trainium. The goal is to remove a major barrier: changing the hardware without rebuilding the application.

The key mechanism is a configuration change. Amazon says a PyTorch application can run on Trainium by changing just one line, with no code rewrite required. TorchNeuron provides the software support that connects PyTorch-based applications to Amazon’s custom AI chip. This makes the move resemble selecting a different hardware target within the development setup.

The approach is not unique to Amazon. The article notes that PyTorch users can also port applications to AMD GPUs or Apple’s chips with a simple configuration change. Amazon’s distinction is that Trainium is paired with the wider AWS cloud. That combination is meant to support startups from prototypes through production, not merely provide another chip destination.

CUDA is Nvidia’s software infrastructure for running applications on Nvidia GPUs and other Nvidia hardware. Nvidia spent years making CUDA support AI applications written with PyTorch. Because PyTorch is used for building, training, and running models, smooth hardware support can strongly influence which chips developers choose.

The mechanism is ecosystem convenience. Developers can write applications in PyTorch and rely on CUDA to connect those applications to Nvidia hardware without handling every hardware-specific detail themselves. The article describes Nvidia’s hardware as deeply embedded in PyTorch, essentially by default. That software connection helped Nvidia become the default hardware choice for many AI applications.

This position is now being challenged. Amazon says it wants PyTorch to become less centered on Nvidia and plans to contribute related code to the main PyTorch project. The aim is a more hardware-agnostic framework. If that works, developers could have more hardware choices. However, the article notes that removing Nvidia-specific support could also create more configuration headaches.

AI developers have choices beyond Nvidia GPUs. The article names AMD GPUs and Apple’s chips as alternatives that PyTorch users can reach through a simple configuration change. Amazon’s Trainium is another option through TorchNeuron. Amazon also contrasts AWS with specialized cloud providers, or neoclouds, such as CoreWeave, Nebius, and Firmus, many of which are tied to Nvidia hardware.

The software experience is similar across several alternatives: developers can port PyTorch applications without completely rebuilding them. Amazon’s added argument is about the surrounding cloud. AWS says startups can move from prototype to production using infrastructure already available on its platform. Trainium also handles both training and inference, which AWS says can improve economics across a complete workload.

The comparison remains unsettled. AMD and Apple offer different hardware destinations, while neoclouds offer specialized computing services. Amazon claims AWS provides broader infrastructure and potentially better overall value, even without the cheapest per-token price. The article notes that developers will have to see whether this creates real savings or more configuration work.

If PyTorch becomes less tied to Nvidia, developers may be able to choose among more chips without rewriting their applications. That could help competing hardware companies and cloud providers attract AI workloads. Greater competition could lead to more affordable options, although the article presents that outcome as a possibility rather than a certainty.

Amazon plans to contribute code to PyTorch itself. The company says it is removing code that ties the framework to specific chips and making PyTorch more hardware-agnostic. In practical terms, developers could select Trainium, AMD GPUs, Apple’s chips, or Nvidia hardware through a less Nvidia-centered software framework. That would weaken the software lock-in that currently favors Nvidia.

The trade-off is uncertainty. PyTorch’s purpose includes hiding the difficult details of running AI code on hardware. Pushing Nvidia out could give developers more choice, but it might also create configuration headaches. The article says the larger consequence could be a real choice of hardware, with possible effects on prices and Nvidia’s position as the default provider.

Training is the process of teaching an AI model. Inference is the process of using the finished model, such as when it answers a user’s question. These are different stages of an AI application’s workload. Amazon says Trainium can handle both, giving startups one chip platform for training and inference.

The cost argument comes from using one platform across the business. AWS says a shared chip and close integration with cloud infrastructure can improve economics across the full workload. A startup may avoid managing separate hardware choices for teaching a model and serving its results. Amazon says this can save money even when AWS does not have the cheapest per-token price.

Deepgram is Amazon’s example. The AI voice provider moved to Trainium using native PyTorch support, and AWS says it processes 70% more audio per dollar at production-grade accuracy. That claim illustrates Amazon’s price-performance argument. The article presents the broader result as a promise: one platform could make both development and production more economical.