NVIDIA releases Dory1.0-4B, a looped open model that trades extra passes for accuracy

What NVIDIA released

NVIDIA published a new small language model on Hugging Face on 16 October 2026 under its Nemotron-Labs label: NVIDIA-Nemotron-Labs-Dory1.0-4B. It has about 4 billion parameters, reads and writes English text, supports a context window of up to 256,000 tokens and is released under the OpenMDW-1.1 licence, a permissive model licence that allows commercial use and places no conditions on what users do with the model's outputs. The model card marks it as generally available and ready for commercial or non-commercial use.

What makes Dory unusual is not its size but its shape. It is a looped, or latent-recurrent, model: instead of stacking dozens of distinct layers, it runs one shared block of layers several times for every token it produces. The number of passes is a setting chosen at inference time, so the same downloaded checkpoint can run lean with one loop or spend more compute with two or four, without retraining and without switching to a bigger model.

The release arrived as model cards rather than a launch event. NVIDIA posted four checkpoints the same day, BF16, MXFP8, NVFP4 and a Q5_K_M GGUF file for llama.cpp, and its company blog feed carried no post about the model when AI Lookout checked on 17 October. By then the BF16 repository showed a few hundred downloads.

How a looped model works

A conventional transformer passes each token through its layers once, top to bottom, so the only way to get more thinking per token is a larger model, and the usual way to get more thinking per answer is to generate more reasoning text. Looped models take a third route. They reuse the same weights repeatedly, refining a hidden representation in place before committing to the next token.

Dory splits its stack into three parts. An input block of 8 layers turns embeddings into a starting state. A recurrent block of 20 layers is then applied k times, and before each pass the original input signal is added back in, so every pass sees the prompt's original representation as well as the refined one. An output block of 8 layers turns the final state into token predictions. With one loop the model runs 36 layers per token; with four loops it runs 96, while the weights on disk stay the same size.

Under the hood, the published configuration describes a normalized-GPT backbone with a hidden size of 2,560, 16 attention heads sharing 4 key-value heads, a 131,072-token vocabulary, and attention layers that alternate between a 1,024-token sliding window and full attention. NVIDIA says it pre-trained the model on 6 trillion tokens, using its 4-bit NVFP4 number format for roughly the first 4.5 trillion before switching to BF16, then extended the context over another 300 billion tokens. Post-training used supervised fine-tuning followed by multi-environment reinforcement learning with GRPO across math, code, science, tool use and conversation tasks.

What NVIDIA's benchmarks show

The model card reports scores at one, two and four loops alongside Qwen3.5-4B, Qwen3.5-9B, Gemma 4 26B A4B and Qwen3.6-35B-A3B, all measured by NVIDIA under the same harness. On SWE-bench Verified, the software-repair benchmark, Dory goes from 50.3 with one loop to 59.5 with two and 61.8 with four. Qwen3.5-4B scores 54.7 and Qwen3.5-9B scores 61 in the same table, so on this test four loops let the 4B model roughly match a model more than twice its size.

The pattern repeats in places but not everywhere. On IFBench, an instruction-following test, Dory reaches 79 at four loops against 59 for Qwen3.5-9B. On the AA-LCR long-context test it scores 73, above every comparison model listed. On Terminal-Bench 2.1, though, four loops reach 22.2, below Qwen3.5-9B at 24 and far below Qwen3.6-35B-A3B at 43. Knowledge-heavy tests barely move with looping: AA-Omniscience accuracy stays at about 10 across all three settings, below Qwen3.5-4B's 15, and GPQA Diamond rises only from 69.2 to about 72, still under Qwen3.5-4B's 77.7.

That split is what the architecture would suggest. Extra passes give the model more computation to work a problem through, but they do not add facts it never stored. NVIDIA's own guidance follows the same line: most of the gain comes from the second loop, with smaller improvements after that, and four loops are suggested mainly for agentic coding work such as SWE-bench and Terminal-Bench.

There is a second claimed benefit. NVIDIA's chart shows that more loops also mean fewer generated tokens: with four loops, Dory used half as many output tokens per attempt on Terminal-Bench 2.1 and 40 percent as many on GDPval-AA-V2 as it did with one. If that holds up, part of the extra compute spent inside each token is paid back by shorter answers.

Memory, formats and speed

A known cost of looping is memory. Even when weights are shared, a naive implementation keeps a separate attention cache for every pass, so cache memory grows with the loop count. Dory offers what NVIDIA calls last-loop KV caching, in which each recurrent attention layer keeps a single cache that every loop overwrites. NVIDIA says this keeps cache memory at the one-loop size for any loop count, and that accuracy in this mode stays close to the default per-loop mode with nearly unchanged output lengths.

The quantized checkpoints target NVIDIA's own hardware. On a DGX Spark desktop system, NVIDIA reports that the NVFP4 version, with 4-bit weights and 16-bit activations, ran 2.17 times faster than BF16 on a 2,560-token prompt with a 300-token answer and 2.44 times faster on a short prompt with a 1,024-token answer. The 8-bit MXFP8 version was 1.69 and 1.84 times faster on the same tests. The GGUF export is 3.67 GB and NVIDIA publishes a SHA-256 checksum for it; the card notes that its benchmark results are for BF16, not the quantized files.

How to run it today

Running Dory is not yet a one-line install. Support in vLLM is still an open pull request, vllm-project/vllm #60149, and the model card tells users to install vLLM from that branch. Support in llama.cpp lives in ggml-org/llama.cpp #30306, also unmerged when checked, and the card's Apple Silicon recipe builds from a pinned commit on a fork. Once running, both servers expose an OpenAI-compatible API.

The loop count is set when the server starts: an n_recurrent_loops override in vLLM, or a dory.num_loops value in llama.cpp, which accepts one to four. NVIDIA recommends two loops as the default, one when compute per token matters most and four when accuracy matters more. It suggests a temperature of 1.0 and top-p of 0.95, says reasoning is always on in vLLM, and lists tool calling through the Qwen3 Coder parser. Supported hardware in the card is NVIDIA Hopper and Blackwell, with H100, B200, B300 and DGX Spark listed as test systems.

Where Dory fits among small open models

Dory lands in a crowded week for compact open models. A day earlier NVIDIA released Nemotron 3.5 Super VL and Bolt, where Bolt relies on native 2-bit quantization to fit a 74B model into 24GB. PrismML's Ternary Bonsai 2 takes the compression route further with weights stored as -1, 0 or +1, and Liquid AI's LFM2.5-24B-A2B keeps only about 2 billion parameters active per token. All of them try to squeeze more capability out of a fixed memory budget.

Dory's bet is different: rather than making each weight cheaper, it uses the same weights more than once and lets the operator decide how many times. The idea of matching compute to the job also runs through Respan's Span-02 decision model, which answers by a deadline, and NVIDIA's own research on faster decoding such as ReplaySSM. The practical appeal is simple: one 4B download that can act like a fast small model or a slower, more careful one depending on the request.

Limits and open questions

Every number in this article comes from NVIDIA. The comparison table was run on NVIDIA's own harness, the Artificial Analysis index figures on the card are labelled internal estimates that may differ from official scores, and one comparison number, Qwen3.6-35B-A3B's SWE-bench score, is a reported figure rather than NVIDIA's own run. Several benchmarks were run only once. No independent group had published results by the time we checked, and AI Lookout did not run the model.

There are practical gaps too. Dory is English-only. Its knowledge scores are weak, so it is a poor fit for factual question answering without retrieval. Latency is not free: four loops mean about 2.7 times as many layer passes per token as one, and NVIDIA itself advises measuring end-to-end latency and cost on real workloads because shorter outputs may or may not offset the extra work. Until the vLLM and llama.cpp changes are merged, running it depends on unreleased branches. The card's training-data notes also describe filtering teacher-model reasoning traces that leaned toward specific political narratives, a reminder that distilled small models inherit choices made upstream.