What did the Mixtral paper specify?

Albert Jiang, Alexandre Sablayrolles, Arthur Mensch, and colleagues describe the model in Mixtral of Experts. Each layer has eight experts.

For every token, the router picks two, processes the current state with those experts, and combines their outputs. The model was trained with a 32,768-token context.

The authors write that it outperforms or matches Llama 2 70B and GPT-3.5 on the benchmarks they evaluated, with the largest gaps they report on mathematics, code, and multilingual tasks. That is their comparison, on their tests.

The instruct checkpoint was fine-tuned with supervised examples and Direct Preference Optimization. The paper says it surpassed GPT-3.5 Turbo, Claude 2.1, Gemini Pro, and Llama 2 70B Chat on the human evaluations they ran.

Those names are the 2024 comparison set in the paper, not a list of today’s models. The general routing idea is in mixture of experts explained.

Why the active count is not the file size

The paper says using a subset of parameters can make inference faster at small batch sizes and raise throughput at large ones. All eight experts still exist in the checkpoint.

A host has to be able to supply the expert the router selects. Apache 2.0 on this release is a grant to use the released weights.

It is not, by itself, the Open Source Initiative’s test for training data and training code.

Common questions

Does 8x7B mean eight separate 7B models?

The paper’s description is one model whose feed-forward block chooses among eight expert groups, with two selected per token. The 47B and 13B figures are the ones the authors use for total reach and active compute.

Does the paper tell you the cheapest way to serve Mixtral today?

No. It does not measure your hardware, precision, or batch size. Compare the checkpoint you actually load.

Sources & further reading

  1. Mixtral of Experts ↗
How this story was made

Written by Kristian Kostov with AI assistance and checked against the linked sources. Company performance claims are attributed to the company. Analysis reflects AiLookout’s interpretation; we have not independently tested the products discussed. Cover photography is illustrative and does not depict the specific announcement or product.

Our editorial standards
Back to all stories