LangChain adds ScaleDown’s constrained decoding models to Deep Agents for context work

What did ScaleDown and LangChain announce?

LangChain’s blog post Building Long-Running Agents with ScaleDown and Deep Agents, dated 13 October 2026 and co-written by LangChain’s Sydney Runkle and Srimanth Tangedipalli with ScaleDown’s Neal Patel, announces two things. First, a new Python package, langchain-scaledown, that plugs ScaleDown’s models into LangChain agents and the Deep Agents harness. Second, ScaleDown is now a direct model provider in the LangSmith LLM Gateway at a /scaledown path.

The pitch is about the unglamorous work inside long-running agents. An agent that has been coding for an hour carries dozens of file reads and tool outputs it no longer needs word for word. A web fetch can return tens of thousands of characters when one paragraph matters. Before an agent files a support ticket, it needs an order number copied exactly, not paraphrased. LangChain’s argument is that every one of those steps is a decision, and that paying a frontier model to make each one is slow and expensive at the volume a long agent run produces.

ScaleDown’s answer is a set of small models built for one task each. The general-purpose model stays in charge of planning, writing code and drafting replies, and the small models handle the context chores around it.

What is a constrained decoding model?

A normal LLM writes any string, one token at a time. A constrained decoding model is trained for a single task and is only allowed to return the answers that task permits: a label from a list you supply, a probability for yes or no, or spans copied from the input. Because the output space is fixed, the model can be much smaller, and it does not have to generate long text to say something short.

ScaleDown’s Classify documentation describes the mechanics. You send text and a set of labels, each with a rubric written as a yes-or-no question. The model scores how strongly the text answers yes to each rubric and normalises the results with a softmax so the scores sum to 1.0, returning the highest as top_label. That gives you a number you can threshold, which a free-text answer does not.

The same documentation offers a second, Jev-compatible Decisions endpoint at /v1/scaledown that accepts a state and typed questions (choice, yes/no, and ordered score scales). ScaleDown says porting from TypeSafe’s Jev Decisions API is mostly a change of base URL and auth header, though its endpoint accepts only its own classify model and reads only text or document fields from the state.

The four models: Summarize, Compress, Extract and Classify

ScaleDown lists four models. Summarize writes an abstractive summary of long text, with sampling settings fixed for faithful output and a system message available to set format, language or focus. Compress cuts a block of context down to the parts relevant to a question, and the docs pitch it for RAG chunks, long chat histories and reasoning traces. Extract pulls fields you describe in plain English, returning each value with a confidence score, character offsets and up to 500 characters of surrounding text on each side. Classify scores text against your labels.

According to LangChain, every model takes up to 1M tokens of input, so a full contract or an hour of agent history fits in one call without chunking. Extract, Summarize and Classify also accept images and scanned PDFs, so a screenshot or a scanned invoice can go straight in. ScaleDown’s Extract overview adds that fields which are really decisions, such as whether an account is past due, can be constrained to a fixed set of choices and are then routed to the classification CDM rather than treated as spans.

Classify can also return a one-sentence reason for each decision. LangChain highlights that for debugging: when an agent stops a run or routes a ticket because of a label, the reason shows why.

How the Deep Agents middleware uses them

The integration ships three ready-made middlewares. ScaledownSummarizationMiddleware replaces Deep Agents’ built-in summarizer. It uses the same trigger and keep options (for example, summarize when history reaches 80% of the agent model’s context window and keep the newest 10%) but sends the older messages to ScaleDown, which returns intent, summary, artifacts and next steps. If a ScaleDown call fails, history is left untouched and the agent keeps running. That is the same problem we described in our explainer on context compaction for agents, moved onto a cheaper model.

ScaledownToolCompressionMiddleware adds an optional compression_query argument to chosen tools. When the model only needs part of a result, it fills in a query such as "refund policy for EU orders"; the middleware strips the argument before the tool runs and compresses the output afterwards. LangChain says one traced example cut a docs page by almost 90%, from 18,402 to 3,110 characters, and the compressed result carries a note telling the model it can call the tool again without compression for the full page.

ScaledownExtractionMiddleware reads only the user’s messages, never the agent’s replies, and saves fields such as order_id, error_message and customer_email into agent state, where tools can read them. A field can also be a set of labels answered by Classify, so an issue can be tagged billing, technical or account. LangChain’s fourth example is a hand-written circuit breaker: after each model call it sends history to Classify with progressing, looping and drifting labels, and ends the run when a non-progress label scores above 0.8.

What do ScaleDown’s benchmarks claim?

ScaleDown’s benchmark methodology scores each model on one public dataset: CLINC OOS accuracy for Classify, CUAD F1 for contract extraction, QMSum ROUGE-L for meeting summaries, and FinanceBench F1 for Compress, where the compressed context is passed to a downstream LLM and the final answer is graded. Cost is compared per 1,000 requests and latency per task.

On its OpenAI comparison page, ScaleDown reports 8.72% higher average accuracy than the mean of GPT-5.5, GPT-5.4 Mini and GPT-5.4 Nano, with per-task gains of 5.40% on Classify, 14.40% on Extract, 8.83% on Summarize and 6.23% on Compress, and about 2.4x lower latency. Its Classify call takes 647 ms against 3,592 ms for GPT-5.5 and 694 ms for GPT-5.4 Mini. The benchmarks overview lists similar head-to-head margins against Anthropic, Gemini, Grok, DeepSeek and Qwen lineups, with speed-ups from 2.2x to 13.9x.

LangChain quotes narrower cost figures from the same benchmarks: Summarize 41x cheaper and 2.7x faster, Compress 138x cheaper and 1.8x faster than the OpenAI average, both with higher accuracy. The 200x faster and 467x cheaper headline is ScaleDown’s broader claim for these narrow tasks. None of these numbers come with an independent replication in the material reviewed.

How it fits the decision-model wave

ScaleDown is entering a crowded field. Since TypeSafe’s Jev decision API and its $870M Series A, vendors have shipped one structured-decision model after another, including Subquadratic’s 384K-token SubQ Snap and open-weight families scored on JevBench. What distinguishes ScaleDown here is less the classifier itself than the bundle: summarization, compression and extraction next to classification, and an integration that drops into an existing agent framework.

LangChain frames it as "cheap by default, frontier on exception": use the large model for open-ended work and small models for the high-volume steps around it. That pattern carries a known risk. A USC preprint we covered found that typed decision models used as agent gates can fail open, so a circuit breaker or router built on a classifier is only as safe as its threshold and fallback behaviour.

Access, gateway support and cost

Getting started, per LangChain, is a uv pip install of langchain-scaledown plus a SCALEDOWN_API_KEY environment variable, and new ScaleDown accounts get 50M free tokens. ScaleDown’s own introduction page, read the same day, still says to contact its sales team for an API key, so self-serve sign-up may vary. The API base URL is api.scaledown.xyz.

Teams using the LangSmith LLM Gateway can have an admin store the ScaleDown key once as a workspace provider secret, then point langchain-scaledown at the gateway’s /scaledown path and authenticate with a LangSmith key. Each ScaleDown call then appears as a LangSmith trace and counts toward the same spend and rate limits as other model calls, which matters if finance teams cap agent spend centrally.

Pricing per token was not stated in the LangChain post or the documentation pages inspected; ScaleDown points to cost tables on its own site. Some agent model names in LangChain’s code samples, such as openai:gpt-6-sol, are illustrations of where the agent’s own model goes, not requirements.

What is still unproven or limited?

Every accuracy, latency and cost figure here is ScaleDown’s, on one public dataset per task. CLINC OOS, CUAD, QMSum and FinanceBench are reasonable choices, but a single dataset per task says little about your tickets, contracts or logs. Several comparison averages include smaller or older models, which flatters the margins, and the docs report percentages without saying whether they are relative gains or percentage points.

Summaries and compression are lossy by design. If a summarizer drops the one constraint a coding agent needed, or Compress trims the clause that answered the question, the agent fails quietly. LangChain’s design mitigates some of that (compressed results say how to fetch the full output, and extraction never reads the agent’s own words), but teams will still need their own evaluations before trusting it in production.

Finally, moving context work to a third-party API adds a dependency and a data flow: full histories, tool outputs and scanned documents leave your stack for ScaleDown. The materials reviewed do not describe data retention or self-hosting options. The integration is a useful option for agent builders, not a measured guarantee of cheaper or better agents.