LightOn releases ColGrep-Agent-2B to offload code search from Claude Code

What did LightOn release?

LightOn, the Paris-based AI company behind the ColGrep code-search tool, published ColGrep-Agent-2B on the Hugging Face blog on 12 October 2026. The model is a 2-billion-parameter code-localization agent: given a question about a repository, it searches with ColGrep, reads promising files, follows references and returns the relevant file paths and line ranges along with the code itself.

The weights are on Hugging Face under the Apache 2.0 licence as lightonai/colgrep-agent-2B, with a GGUF build for local inference. The model card lists openbmb/MiniCPM5-2B as the base model. The research behind it is a 4 October arXiv paper, “Small Agents with Semantic Search: Efficient Multilingual Code Localization”, by Maxence Lasbordes, Aarush Sinha, Raphael Sourty, Amélie Chatelain and Djamé Seddah.

Three names are easy to confuse. ColGrep is the semantic code search engine. ColGrep-Agent-2B is the new model. And colgrep --agent is the command that runs that model with ColGrep and a read-only terminal. The agent ships inside the existing open-source ColGrep tool rather than as a separate product.

How does a 2B code-search agent help Claude Code?

Before a coding agent can fix a bug, it often spends several turns working out where to look: it greps for a name, opens a file, follows a reference and tries another search. Each of those steps runs through the frontier model, and the growing conversation is sent back to the API on every turn. LightOn’s idea is to hand that exploration to a small model running on the developer’s own machine.

When Claude Code is connected, it can pass a question such as “where is this behaviour implemented?” to colgrep --agent and get back a short list of locations, each with the name and code of its enclosing function, so it can often answer without opening the files. Under the hood, ColGrep uses LateOn-Code-edge, a 17-million-parameter multi-vector retrieval model that keeps a vector per token instead of one per function. That late-interaction approach, also behind Perplexity’s pplx-embed-v2-late, lets a query about “custom attributes” find code named fieldDefinition.

The pattern matters beyond LightOn. Coding agents are getting more expensive to run as tasks get longer, which is why work like Together’s Link for coding agents also looks for ways to cut the tokens a frontier model burns. ColGrep-Agent-2B is a concrete, open example of delegating one well-defined subtask to a specialised small model.

How much did it save, and what did it cost in accuracy and time?

LightOn ran Claude Code with Sonnet 5.5 on 52 code-search questions across 13 repositories that were not used in training. Each question ran three times in each of four setups, 624 sessions in total: built-in grep only; plain ColGrep semantic search; ColGrep-Agent; and a Balanced mode that lets Claude choose plain ColGrep for names and patterns or the agent for behaviour questions. Claude Opus 5.5 graded answers against reference locations without knowing which setup produced them.

With the agent, Claude Code averaged 93,000 API tokens per question against 142,000 with grep, a 34.3% saving with a 95% confidence interval of 28–40%, and it used fewer tokens on 88% of questions. Cache-aware API cost fell from 7.3 cents to 4.4 cents per question. Turns fell from 5.5 to 3.9, because each extra turn resends about 23,000 tokens of accumulated context. Tokens generated by the local 2B model are not counted, since they are not billed.

Accuracy was 80.8% with the agent versus 83.3% with grep. LightOn says the 2.6-point drop is not statistically significant, but it also notes the confidence interval, −10.9 to +5.1 points, is wide enough that a meaningful drop cannot be ruled out. The Balanced mode matched grep’s 83.3% while saving 21% of tokens.

Time went the other way. End-to-end question time rose from 13.0 seconds with grep to 21.6 seconds with the agent; a local agent call took 7 seconds at the median and 22 seconds at the 90th percentile on LightOn’s GPU server. Savings also grew with repository size, from 19% below 50,000 lines to 40% between 150,000 and 600,000 lines.

How was ColGrep-Agent-2B trained?

LightOn trained the model in two stages starting from MiniCPM5-2B. First came a short supervised fine-tuning cold start on trajectories generated by Qwen3.5-27B inside the same ColGrep harness, with grep calls rewritten as equivalent ColGrep commands. Then came reinforcement learning with a variant of Group Relative Policy Optimization (GRPO) for 500 steps, sampling 16 trajectories per issue.

The harness is deliberately minimal: three tools (colgrep for search, a read-only terminal, and finish to return locations) and a 10-turn budget. The reward averages file-level and line-level F1 against the lines a real fix touched, with a penalty for extra turns after the second, a tiny penalty for falling back to grep, and penalties for never searching or not submitting. Training data came from subsets of nebius/SWE-rebench and internlm/SWE-Fixer-Train-110K plus scraped repositories, so each example pairs an issue with the commit that fixed it.

On code-localization versions of SWE-bench Lite and Multi-SWE-bench Flash, LightOn reports that the 2B agent reaches 76.18 file F1 and the highest line F1 in its comparison, 19.92, on SWE-bench Lite, against models more than ten times larger run in the same harness, such as Gemma 4 31B and Qwen3.5-27B. On the Semble code-search benchmark it returned 1.7 files per question with 0.744 precision at 10, versus 9.8 files for plain ColGrep. These are LightOn’s own evaluations.

How do developers install and use it?

Installation is one command: brew install lightonai/tap/colgrep followed by colgrep --install-agent, which downloads the default Q8_0 GGUF of about 2.5 GB and prepares a prompt cache. Shell installers for macOS, Linux and Windows and a Cargo build are also listed in the ColGrep README. No Python environment, inference server or API key is needed.

After indexing a project once with colgrep init, developers can ask questions directly, for example colgrep --agent "where are retries configured", with JSON output for scripts. The index stays on the machine and updates as files change. LightOn says inference uses Metal on Apple Silicon and Vulkan on NVIDIA, AMD and Intel GPUs, falling back to the CPU; on an M3 Pro a question takes around 20 seconds with 3–4 GB of peak memory.

Integrations install with colgrep --install-claude-code, --install-codex or --install-opencode. For Claude Code, the integration uses hooks that suggest the search tool in the agent’s context, and LightOn’s headline numbers come from that hook mode. Teams already running OpenCode 2.0 or Codex can try the same delegation without changing models.

What are the limits of these results?

The evidence is LightOn’s own, on a benchmark it built: 52 questions, four per repository, with an LLM writing the harder cross-folder questions and another LLM grading answers. LightOn is open about this, and it reports confidence intervals and per-repository noise, but no independent group has reproduced the savings yet.

The test also measures code search, not coding. LightOn explicitly says the results are for code-search questions rather than end-to-end tasks, so it is unknown whether a 34% cut in search tokens survives a full bug-fix session, or whether a slightly less accurate search sends the frontier model down wrong paths later. Latency rises, and the local model needs a few gigabytes of memory and ideally a GPU.

For teams paying per token on large repositories, the practical reading is that the Balanced mode is the conservative choice, matching grep’s accuracy with a 21% saving, while the full agent trades some speed and possibly some accuracy for the biggest cut in API spend.