An Interview with Microsoft’s Scott Guthrie on Azure’s New AI Chip, ‘Insatiable’ AI Demand and Today’s AI Model Price War - Key Context by Tae Kim

Microsoft announced that its Azure Maia AI chip program will expand to multi-gigawatt-scale deployments starting next year. Maia 300 is the next major generation, following Maia 200, which is already in production. The announcement matters because Microsoft wants more control over AI costs, capacity, and performance across its cloud services.

Maia 300 will support Microsoft’s first-party offerings, as Maia 200 already does. It will also power third-party services through Azure AI Foundry and Microsoft’s API service. Customers may therefore use Maia chips indirectly when they access Microsoft-hosted AI models and applications.

The deployment will ramp over time, and Microsoft has not provided an exact date for reaching multi-gigawatt scale. Guthrie said supply-chain orders are already large enough for suppliers to see Maia 300 “in earnest.” Maia 400 and Maia 500 are planned successors.

Maia 300 is the next major version of Microsoft’s Azure Maia AI chip program. It is designed specifically for AI workloads, especially the inference stage that produces model responses. Microsoft is building it to improve throughput and reduce the energy and total cost required to serve AI applications.

A general-purpose GPU is a programmable processor used across many kinds of computing, including AI. Maia 300 is a more specialized accelerator: its design can be tuned around the operations and data movement common in AI models. The article does not provide detailed architectural specifications, so the comparison rests on Microsoft’s stated performance target rather than disclosed chip details.

Microsoft says Maia 300 will deliver 40% more tokens per megawatt than leading GPU-based competitors. It will still use Nvidia and AMD chips, alongside Maia. The goal is not to replace every GPU, but to add another silicon option for Microsoft’s growing Azure capacity.

Microsoft described its Maia 300 deployment as multi-gigawatt scale, beginning next year. It did not state the final number of gigawatts or the exact timetable for reaching that level. The plan will ramp as supply-chain capacity and deployments increase.

A watt measures the rate at which equipment uses power. A gigawatt equals one billion watts, so multi-gigawatt scale means several billion watts of power for the relevant Maia 300 infrastructure. That figure describes power capacity, not the number of chips. The actual chip count would depend on each chip’s power use and the surrounding data-center equipment.

The scale signals a major expansion beyond Maia 200, which already runs in healthy volumes. Guthrie said Microsoft is ordering many Maia 300 chips, making the program visible to suppliers. Microsoft also plans Maia 400 and Maia 500, showing that the deployment is intended as an ongoing program.

Tokens per megawatt is an energy-efficiency measure for AI serving. A token is a unit used to process text, while a megawatt is one million watts of power. The metric asks how much model output a system can generate from a fixed amount of electricity. Higher performance means better energy productivity.

Microsoft says Maia 300 will deliver 40% more tokens per megawatt than leading GPU-based competitors. For example, if a competing system produced a certain amount of output using one megawatt, Maia 300’s target would be 1.4 times that output under comparable conditions. The article does not specify the models, workloads, or testing setup behind the comparison.

The measure matters because electricity is a major constraint in large AI data centers. Better output per unit of power can reduce inference costs, support more customer requests within available capacity, and improve total cost of ownership. Microsoft links higher throughput per watt directly to the amount of Azure capacity it can deliver.

Microsoft is developing Maia partly to reduce the cost of inference, the computing needed to generate responses from trained AI models. Lower inference costs can improve Azure’s total cost of ownership and give Microsoft more flexibility in pricing its AI services. The company also wants stronger output per watt, not just cheaper chips.

The mechanism is straightforward: Microsoft owns the cloud infrastructure, so it can deploy Maia under the services customers use. Maia 300 is intended for Microsoft’s first-party offerings and for third-party customers using Azure AI Foundry and Microsoft’s API service. Those customers may benefit from the chip’s performance without buying or managing the hardware themselves.

Microsoft says it can pass savings to customers and deliver more capacity when costs and power use fall. The article does not promise specific price cuts. Azure will still use Nvidia, AMD, and Intel products, so the effect will depend on workload, supply, deployment speed, and competition among available chips.

Microsoft is pursuing multiple chip sources because its cloud business must serve many models, applications, and customers at global scale. Its own Maia and Cobalt chips can be tailored to Microsoft’s needs, while outside suppliers provide established products and additional capacity. This reduces dependence on one silicon program.

The article gives a direct example. Microsoft will use Maia for first-party offerings and for third-party services, while continuing major partnerships with Nvidia and AMD on AI chips. On the CPU side, Cobalt is ramping sharply, but Microsoft will still buy many CPUs from AMD and Intel. Even Axiom is partnering with Microsoft on its own first-party silicon program.

Guthrie calls this approach “multi silicon,” matching Microsoft’s “multi model” strategy. Microsoft supports Axiom, Paradox, and its own models, while offering infrastructure in countries seeking sovereignty. The company therefore needs flexibility across performance, supply, security, compliance, and customer needs rather than one universal chip choice.

AI training is the process of adjusting a model using large amounts of data so it learns patterns and can produce useful outputs. Inference happens after training: the model processes a prompt and generates a response. The article focuses mainly on inference economics, especially the cost of producing tokens for Microsoft’s services.

Chip throughput determines how many calculations a system can perform. Memory bandwidth determines how quickly the chip can move model data and intermediate results. Electricity powers both computation and data movement. If any of these limits throughput, the system may produce fewer responses or tokens per second and require more hardware for the same workload.

That combination affects the cost of an AI response because cloud providers pay for chips, power, and infrastructure while serving requests. The article does not explain memory bandwidth in detail, but it links inference cost to throughput per watt and tokens per dollar. Maia 300’s stated 40% advantage is therefore an efficiency claim, not a promise about every workload.