Waterloo’s Project Greenhouse open-weights a from-scratch Gaggle reranker

What is Project Greenhouse’s first milestone?

The authors define agentic search as models using search tools in loops and argue that fully open, sovereign components can be built with modest compute. Their first delivered piece is a pointwise decoder-only reranker—score each (query, document) pair, then reorder a first-stage list.

They deliberately skip embedding-model pretraining and full agent RL as a first step, citing faster iteration when candidates are fixed. Commercial rerankers from cloud vendors are cited as evidence the component is useful alone.

Pointwise reranking sits after first-stage retrieval; for embedding-side context see vector search explained and our note on Perplexity’s late-interaction embedders.

The paper’s framing matters for how you cite it: Greenhouse is presented as a multi-year programme toward fully open agentic search stacks, and Gaggle is explicitly milestone one—a reranker—not a finished tool-using agent. That is why the Hub card and the arXiv abstract both talk about pointwise relevance scoring rather than planner loops or browser tools. Treat “sovereign agentic search” as the project banner and Gaggle as the downloadable component you can actually score today.

How was Gaggle trained?

Pretraining used Andrej Karpathy’s nanochat depth-34 recipe on ClimbMix to produce gaggle-nanochat-pretrained-climbmix-20260924 (~3.29B). Fine-tuning kept two LM head rows for the tokens “ true” and “ false,” yielding about 3.22B parameters in the reranker.

Relevance training used RLHN-250K with localized contrastive estimation for one epoch. The released headline model is a parameter soup of four runs (seeds/hardware variants) averaged at weight 0.25—checkpoint name gaggle-reranker-20261005.

The paper stresses the bulk of experiments used a handful of GPUs; SFT typically one or two GPUs, with pretraining on an 8×H100 server for the heavier phase. Those resource lines are the authors’.

ClimbMix pretraining followed by RLHN-250K fine-tuning is the authors’ recipe; the soup of four runs is their variance-reduction step for the headline checkpoint name. If you train a sibling model, expect to re-pin ClimbMix revision hashes, nanochat depth settings, and the exact LCE schedule before claiming parity. The 3.22B active reranker head that keeps only the “ true” / “ false” vocabulary rows is a deliberate compression of a larger pretrained LM into a binary relevance scorer—useful when you want open weights without importing a third-party backbone’s license stack, still subject to ClimbMix’s own data terms.

Which scores are the authors’, and what is open?

On the Hub card, the soup’s mean nDCG@10 reranking BM25 top-100 is 0.641 across TREC DL19–23 and 0.547 across seven BEIR sets. Individual DL rows include 0.757 (DL19) and 0.480 (DL23). Four BEIR sets are called out-of-domain relative to RLHN-250K.

Weights are MIT on Hugging Face; ClimbMix is CC-BY-NC-4.0 and RLHN-250K is CC-BY-SA-4.0, so downstream commercial use must still read those data licenses. trust_remote_code is required because the architecture files ship in-repo.

That license split is a concrete instance of the open weights versus open source distinction: MIT weights plus non-commercial pretraining data are not a single green field.

When the authors report nDCG@10 lifts on TREC and BEIR slices, keep the first-stage retriever fixed in your mental model: a reranker’s gain is conditional on the candidate set BM25 (or your dense retriever) already returned. Swapping Gaggle into a pipeline that uses a different top-k or a different first stage can erase or invert those deltas. The MIT mark on castorini/gaggle-reranker-20261005 covers the weight files we opened on the card; it does not automatically clear ClimbMix training-data redistribution questions for every jurisdiction.

Re-check on 11 October 2026: the Hub API for castorini/gaggle-reranker-20261005 still lists four safetensors shards (model-00001-of-00004.safetensors through model-00004-of-00004.safetensors) plus modeling_gaggle.py and tokenizer files under MIT. Ai Lookout did not download the shards or rerank BEIR.

What does “fully open and sovereign” mean here?

The paper models training as a property hypergraph so every artifact and command can be cited. The authors argue that starting from third-party open weights still leaves upstream corpora and recipes opaque.

They scope out general coding and frontier reasoning. Greenhouse is framed as a long-running project; Gaggle is an existence proof for one retrieval component, not the finished agentic search stack.

Contrastive ablations fine-tune other backbones (Gemma-4-E2B, Qwen3.5-4B-Base, MiniCPM, nanochat-d34) with the same SFT recipe; those tables are in the paper for readers who want backbone comparisons.

What should a retrieval team try?

Load castorini/gaggle-reranker-20261005 with the card’s transformers example, score a BM25 top-100 from your corpus, and compare nDCG to your current cross-encoder before changing production.

Respect the 128/512 training truncation if you want numbers comparable to the card; longer contexts are untested per the model card.

If sovereignty is the goal, budget for ClimbMix’s NC license review—not only the MIT weight license.

A practical bake-off is three columns on the same corpus: your production cross-encoder, Gaggle soup, and one of the parent seed checkpoints if you care about soup variance. Log not only nDCG@10 but also latency at your batch size and the fraction of ties the pointwise head produces. If you already follow Waterloo’s Anserini/Pyserini tooling, wire the Hub example before you invent a new serving path—the card’s transformers snippet is the authors’ intended smoke test.

What did we not test?

We did not download the 12.9 GB shards, run trec_eval, or reproduce Table scores. This article reports arXiv 2610.11922 and the castorini Hub cards we opened on 10 October 2026.