Microsoft AI finishes pre-training MAI-Base-2, a 2.8-trillion-parameter model

What Microsoft AI announced on 15 October

Microsoft AI, the unit led by Mustafa Suleyman, published a long technical note titled Frontier Pre-training at Microsoft AI on 15 October 2026. Its headline: the lab has completed pre-training MAI-Base-2, a new base model that it calls state-of-the-art, with 80 billion active and 2.8 trillion total parameters.

This is not a product launch. A base model is the raw result of pre-training, before the instruction tuning, reinforcement learning and safety work that turn it into an assistant. Microsoft says MAI-Base-2 is “currently undergoing post-training” and that it will share more results soon. There is no API, no price and no release date in the post.

What the post does offer is unusual detail for a closed lab: the architecture, the hardware, the length of the training run, the data mix and several engineering tricks. It also says preparations for MAI-Base-3 are already under way. Microsoft frames the work as a step towards owning its full model pipeline, from data to chips, rather than relying only on partners.

Inside MAI-Base-2: 80 billion active parameters out of 2.8 trillion

MAI-Base-2 is a mixture-of-experts model, which means only a small slice of its weights is used for each token. Our mixture-of-experts explainer covers the basic idea. Here, each token is routed to one shared expert plus 7 of 512 routed experts, so about 80 billion of the 2.8 trillion parameters are active at a time.

Compared with MAI-Base-1, which Microsoft released in June with 35 billion active and 1 trillion total parameters, the new model is deeper and sparser. It has 102 layers instead of 78, a hidden dimension of 8,704 instead of 6,656, and an expert layer in every block instead of alternating dense and expert layers. Attention mixes global layers every sixth layer with local ones, and the embeddings are no longer tied.

Two other changes stand out. The optimizer moved from AdamW to Muon for most weight matrices, with AdamW kept for embeddings. And the model is natively multimodal: Microsoft describes an encoder-free vision-language design trained on images from the start, whereas MAI-Base-1 was text only. Training ran for 50 trillion tokens, up from 30 trillion, using an MXFP8 number format.

How MAI-Base-2 compares with other trillion-parameter models

Base models are hard to compare because most labs only release tuned versions. Microsoft uses bits per byte, a measure of how well a model predicts unseen text that does not depend on its tokenizer. Lower is better. The held-out code set is Microsoft AI's own codebase; the STEM, math and QA sets were commissioned from outside vendors to reduce the risk that any model had seen them. Microsoft says it also decontaminated every set against its training data.

On all four, MAI-Base-2 scores lowest: 0.2203 on code, 0.4162 on QA, 0.3653 on STEM and 0.3561 on math. Kimi K3 is second each time. DeepSeek-V4-Pro-Base, which Microsoft calls the most capable open base model, scores 0.2370, 0.4351, 0.3968 and 0.3862. MiMo-V2.6-Pro-RL and Qwen3.8-2.4T-A95B sit in between.

There is a real caveat. Only post-trained checkpoints exist for Kimi K3, Qwen3.8 and MiMo, and Microsoft notes that post-training may affect these scores. The gaps are also small in absolute terms, and bits per byte does not tell you how a model will do on coding agents or reasoning tasks. It says nothing yet about how MAI-Base-2 compares with open-weight releases like Meta's Muse Spark Open, which arrived the same day and was not in the comparison.

The efficiency claim: 4.2 to 12.3 times fewer FLOPs than DeepSeek

The bolder claim is about efficiency. Microsoft says its training recipe is 4.2 to 12.3 times more FLOPs-efficient than DeepSeek-V4-Pro-Base, depending on the evaluation. To get there, it fitted a scaling law of bits per byte against training compute for its own family of models, then estimated how much compute a model in that family would need to match DeepSeek's score, and compared that with DeepSeek's actual compute.

By the same method, Microsoft says the recipe improved 6.3 to 12.7 times on MAI-Base-1 in the four months since June. It credits data quality and modelling choices. The company says it put that saving into training a larger model rather than a cheaper one.

This is an estimate built on Microsoft's own scaling ladder, a set of models of increasing size trained with a fixed recipe. The post describes a technique called source scaling, which repeats each dataset the same number of times at every rung, so small experiments do not favour settings that would overfit at full size. Microsoft shows its ladder predicted MAI-Base-2's final loss. Outsiders cannot check the comparison without the ladder data.

38 days on 16,000 GB200 GPUs

Microsoft trained MAI-Base-2 on 16,000 NVIDIA GB200 GPUs over 38 days in August and September 2026. Average model FLOPs utilisation was 21 percent of the chip's theoretical BF16 peak, and goodput, the share of time training actually progressed, was 89 percent. That run used NVIDIA hardware; Microsoft's own Maia 300 inference chip is aimed at serving models, not training them.

Much of the post is about making sparse layers fast. Microsoft says fixing memory fragmentation and per-expert overhead lifted the utilisation of its expert layers from 24 to 35 percent, which is what made an expert layer in every block worthwhile. A multi-round “dropless” design processes overflow tokens in extra rounds instead of discarding them, keeping memory fixed and avoiding out-of-memory crashes.

Muon, the optimizer, needs whole weight matrices, which clashes with the way memory is split across GPUs. After optimisation, Microsoft says, Muon took 2.1 percent of each training step, or 165 milliseconds, against an estimated theoretical best of 55. Activation gradient clipping, which trims outlier gradients inside the network rather than scaling down the whole batch, gave a 28 percent efficiency gain at the largest test size. It also exposed a preprocessing bug that had cut a licence header off mid-sentence.

Organic data, no distillation, and native vision

Microsoft stresses that MAI-Base-2 is trained on “organic data” and not distilled from other companies' models, and that every training token was processed in-house. Its corpus of unique tokens grew about 36 percent from MAI-Base-1, with a March 2026 knowledge cutoff. Microsoft says its web crawler respects robots.txt and that it excludes sites on the US Trade Representative's Notorious Markets list.

Code is the largest part of the 50-trillion-token mix at 45 percent, followed by vision-only data at 14 percent, web text at 12, PDFs and STEM at 9 each, books at 5, math at 4 and other sources at 2. The vision corpus holds 4.5 trillion unique tokens, covering captions, charts, documents, computer use and even sheet-music reading. With repetition it made up 18 percent of training.

The provenance point is commercial as much as technical. Microsoft says clean data lineage will let it license MAI weights to trusted partners through Foundry, its enterprise model platform, a claim aimed at customers wary of models with unclear training sources.

Why Microsoft is building its own frontier models

Microsoft has relied heavily on OpenAI models across Copilot and Azure. A frontier-scale base model of its own gives it an alternative it controls, from training data to deployment. The lab's June model, MAI-Thinking-1, was built on MAI-Base-1, and Microsoft already sells smaller in-house models such as Decision-1 in Foundry.

The note says owning the full stack lets Microsoft tune models and infrastructure together for the best performance at the lowest cost, “first in our own products and then for Foundry customers.” A model of this size, post-trained well, would give Microsoft a frontier option of its own alongside the partner models it hosts.

What is still missing

Everything here is Microsoft's own reporting. The comparisons use held-out sets that only Microsoft can run, the efficiency figures depend on its private scaling ladder, and there are no results yet on standard benchmarks, agent tasks or safety tests. Those will come only after post-training.

Questions to watch: when a post-trained MAI model built on MAI-Base-2 ships and how it scores in public tests, which partners get weight licences through Foundry and on what terms, and how quickly MAI-Base-3 follows. For now, MAI-Base-2 is a strong signal of Microsoft's intent and an unusually open look at how a closed lab trains at this scale, not something developers can use.