Microsoft unveils Maia 300 inference chip and plans multi-gigawatt Maia deployments from 2027

What did Microsoft announce on 12 October?

In a post on the Official Microsoft Blog, Scott Guthrie, executive vice president of Cloud + AI, said Microsoft will scale its Azure Maia family to multi-gigawatt deployments starting in 2027 and make Maia “the primary inference system for Microsoft products and services”. The post is headed “Azure Maia 200 now in production; Azure Maia 300 coming in 2027”. Rani Borkar and Saurabh Dighe of Azure Hardware Systems published a technical deep dive on the Azure Blog at the same time, and Microsoft’s EMEA newsroom ran a short summary.

The headline claims are two. Microsoft expects Maia 300 to deliver 40% more tokens per megawatt than “leading GPU-based systems”, a comparison it does not name. And in a full 96-accelerator configuration, it says Maia 300 delivers up to seven times the rack-scale token throughput of Maia 200. Tokens per megawatt matters because power, not chip supply alone, is now the limit on how much AI capacity a cloud can add.

Guthrie frames the chip around three workloads: long-context reasoning, latency-sensitive conversation, and sustained, multi-step agentic tasks. Those workloads spend most of their time generating tokens one at a time and reading large key-value caches, which stresses memory bandwidth more than raw arithmetic.

What is inside a Maia 300 accelerator?

Each Maia 300 accelerator has eight HBM4 stacks, giving 288 GB of capacity and 18 TB/s of peak bandwidth within a 950-watt thermal design power. Against Maia 200, Microsoft says that is 2.5 times the memory bandwidth, 33% more capacity and roughly twice the HBM bandwidth per watt. Our explainer on why bandwidth and memory matter alongside compute covers why those numbers can matter more for serving models than peak FLOPs.

The chip is split into chiplets. Compute dies are built on TSMC’s N2P node, while memory controllers and input/output sit on separate dies optimised for bandwidth and cost. Microsoft says this raises the system’s compute FLOPs by 36%, adds 2.5 times the HBM bandwidth and gives 14% more scale-up bandwidth. Memory changes from a uniform layout to a non-uniform (NUMA) one across chiplets, so software can keep data close to the compute that uses it.

The software model carries over from Maia 200. Microsoft calls it SDLA, a software-defined, locally accessed dataflow architecture that separates control from data movement and lets software place, move and convert data explicitly across local SRAM, DMA engines, HBM and the network. Maia 300 adds an enhanced MXFP4 format and sub-3-bit storage formats with hardware conversion, which cut the bytes moved per token and shrink the KV cache. Our guide to what quantization trades away explains why lower-precision formats save memory, and why their effect on quality depends on the model and method.

The vector engine also gets instruction-set additions for inference kernels such as softmax and low-precision casting. Guthrie mentions AI-assisted kernel optimisation, but neither post explains how that works.

How do Maia 300 racks and superpods fit together?

A Maia 300 rack holds 96 accelerators across 24 trays of four, inside a 167-kilowatt power envelope, twice the accelerator density of Maia 200. Each tray carries its own host processor, memory, storage and cooling, so there are no separate host trays. Six high-radix switch chips sit in three liquid-cooled switch trays. Together a rack offers 27.6 TB of HBM and 1.7 PB/s of aggregate memory bandwidth.

The network is Microsoft’s own Ethernet-based AI transport in two tiers. Inside a rack, two accelerators in a tray share direct links, and a switched fabric joins all 96; those links are electrical to save power and cost. An optical second tier connects up to 128 racks into a superpod of 12,288 accelerators. Any accelerator can reach any other without going through a host or a separate network card, with at most three switch hops, and can push up to 750 GB/s to a peer when only some accelerators are crossing racks.

That design lets Microsoft split inference jobs. Prefill, which reads the prompt, and decode, which generates the answer, can run on separately sized groups of accelerators, with the KV cache moving between them over the same fabric. Spare trays in one rack can stand in for failed trays in another. For readers new to the distinction, our explainer on training versus inference sets out why serving models is a different engineering problem from training them, and why a bigger context window does not guarantee a better answer covers the long-context side.

Power and cooling are designed with the building. Liquid cooling removes more than 80% of the rack’s heat, the rack can connect to facility water or reject heat through a heat exchanger in air-cooled halls, and a 50-volt busbar with dual feeds and an automatic transfer switch keeps it running while one feed is serviced. Rack and row telemetry feed back into how Microsoft places workloads.

Where does Maia 200 stand today?

Maia 200 is the production chip today. Guthrie says it is serving Microsoft production workloads including Microsoft Copilot and supporting Microsoft’s MAI team as it develops next-generation models, with GitHub Copilot and Microsoft Foundry to follow by the end of the year. It runs OpenAI models, Microsoft’s own MAI models and open-source model families, and is deploying across the United States, Europe and Australia.

Microsoft says Maia 200 delivers up to 1.7 times more tokens per watt than leading GPU infrastructure for built-in AI features in Teams, Copilot and Cowork. The Verge, which reported the launch, quotes Guthrie calling Maia 200 “the best performing AI infrastructure in our fleet, measured in real world metrics for AI services”. Microsoft’s smaller models are another target: its Decision-1 scorer in Foundry is the kind of high-volume inference workload where cost per token decides whether a product makes sense.

Guthrie says experience running Maia 200 shaped Maia 300, and that the two generations together are meant to support GitHub’s agentic coding, Foundry model serving, inference and data generation for the MAI team, and latency-sensitive Dragon Copilot workflows for clinicians.

What does Maia 300 mean for Nvidia, AMD and Azure customers?

For Nvidia and AMD, the change is in the plan rather than the purchase order. The Verge reads the announcement as Microsoft preparing to rely mainly on its own chips for inference instead of GPUs. Microsoft has not said it will stop buying GPUs; its own blog lists a 20 July post by Guthrie on expanding Azure AI and HPC infrastructure with AMD. Microsoft AI chief Mustafa Suleyman told The Verge: “At that scale we need a platform that we own, and that we co-design with our models.”

Microsoft joins Google, with its TPUs, and Amazon, with Trainium and Inferentia, in betting that a cloud running its own models at huge volume can beat general-purpose GPUs on cost per token. Training remains a different market; Maia 300 is described only as an inference system.

For Azure customers, the effect is indirect. Neither post announces Maia virtual machines or prices; the chip serves Microsoft’s own products, and Foundry is the route by which customer workloads may run on it. If the efficiency claims hold, cheaper inference could show up in Copilot and Foundry pricing or capacity, but Microsoft has not said so. Teams weighing managed hosting against self-hosting will not see a Maia option on a price sheet yet.

What can’t we verify yet?

Almost everything here is a vendor expectation. The 40% tokens-per-megawatt figure compares Maia 300 with unnamed “leading GPU-based systems” on unstated models and settings. The 7x throughput figure is relative to Microsoft’s own previous chip and depends on the full 96-accelerator configuration. We found no independent benchmark or customer measurement, and the hardware does not reach production until 2027.

Microsoft did not give a gigawatt figure beyond “multi-gigawatt”, a deployment timetable by region, or the cost of the programme. The posts reference a technical paper and a Maia 300 microsite with photos and video; we read the two blog posts, the EMEA summary and The Verge’s report, not the technical paper.

Microsoft says it will share more at Ignite 2026 in San Francisco next month (17 to 20 November). The figures and the charts in this article come from the 12 October posts; the photographs show Azure datacenters, not Maia hardware.