IBM's Granite Switch 4.1 runs chat, decisions and speech from one endpoint

What IBM released

IBM's Granite team announced Granite Switch 4.1 in a Hugging Face blog post dated 15 October 2026, written by Kate Soule, Subhajit Chaudhury, Hendrik Strobelt and Luis A. Lastras. Two ready-made checkpoints are available: Granite-Switch-4.1-8B and Granite-Switch-4.1-30B. Both model cards give an Apache 2.0 licence and a release date of 14 October 2026, and list Hugging Face Transformers and vLLM as supported backends.

Each checkpoint is described as a modular model that composes a base model, adapters and an encoder so that "a single hosted endpoint can move between natural-language responses, general and specialized decisions, and speech-to-text." The 8B version is built on Granite 4.1 8B and the 30B on Granite 4.1 30B; both bases have a 128,000-token context window.

The Granite Switch project itself is not new. IBM says it introduced the approach last spring, and the open-source repository on GitHub was created in May 2026. What 4.1 adds is a general-purpose Decision adapter, a Guardian safety adapter and an embedded speech recognizer, shipped together as a "batteries included" model. IBM also offers larger preview bundles in 3B, 8B and 30B sizes that include the complete Granite Libraries adapter set but no audio.

How one checkpoint does three jobs

The trick behind Granite Switch is a technique IBM calls Activated LoRA, or aLoRA. A LoRA is a small set of extra weights that adapts a frozen base model to a task. A conventional LoRA changes how the whole prompt is processed, so switching adapters usually means recomputing the model's working memory of the conversation, known as the KV cache.

An activated LoRA only switches on at a specific point in the prompt, triggered by control tokens in the chat template. Everything before that point is processed by the plain base model, so the KV cache built during an ordinary chat response can be reused when the same request is then sent to a decision or safety adapter. IBM says this lets the model switch between a chat request, a decision request and a spoken request "with blazing fast speeds," helped by the team's recent contributions to vLLM for faster low-rank adapter inference.

Because the adapters are small and share the base weights, IBM says the full bundle takes up "approximately the same amount of VRAM on a GPU as the base model would by itself." In practical terms, a team that would otherwise run a chat model, a separate classifier and a separate speech server could, in IBM's pitch, run one model on one server instead.

The repository README frames it as building models the way software is built from libraries: pick adapter functions for RAG, safety, factuality and other tasks, compose them into one checkpoint and swap or upgrade any of them without retraining the base model. Each adapter function is trained to a fixed input and output contract, such as a score or a decision, with the output format enforced at the token level by IBM's Mellea library.

The Decision and Guardian adapters

The new Decision aLoRA is aimed at the same job as the decision models that have multiplied this autumn, from Typesafe's Jev to Microsoft's Decision-1. According to the model card, it takes a state and a question with labelled options and returns a probability distribution over those option labels in a single forward pass, rather than generating a free-text answer.

IBM evaluated the adapter on the public fast subset of the Decision Index, 1,068 rows in its edition-v0.3 scoring. The 30B checkpoint scored 50.1 overall, with 32.4 for knowledge, 58.6 for language, 49.8 for retrieval, 76.5 for tools and 26.4 for arts. The 8B scored 42.9 overall, with 29.1, 48.9, 49.8, 59.8 and 18.0 across the same areas. IBM says it measured each configuration both locally and on a hosted vLLM endpoint and that the two agreed to within about 0.6 points.

The Guardian Core aLoRA comes from IBM's granitelib-guardian library and detects safety risks including harm, jailbreaking, profanity, violence, sexual content, social bias and unethical behaviour. IBM claims that, on average, it outperforms Jev 1.13, Microsoft-Decision-1, GPT-Luna-Decisions and Cloudflare's Clef-Flash on safety-related decision tasks. The blog points to an evaluation Space for the comparison; AI Lookout has not reproduced it.

IBM's broader argument is that specialists beat generalists on narrow tasks. Its blog says many of the specialised granitelib adapters outperform general decision models on specialised classification tasks, and that because adapters share the KV cache, adding specialists alongside a general Decision adapter "costs almost nothing."

Speech on the same endpoint

The third component is Granite Speech 5.0 TurboCTC, a 470-million-parameter English speech recognizer that IBM describes as the fastest model on the Open ASR Leaderboard. It is a non-autoregressive conformer CTC encoder, which means it emits a transcript in one pass rather than word by word.

On a Granite Switch endpoint, incoming audio is transcribed by that encoder and the transcript is spliced into the chat context in place of the audio, so callers use the same OpenAI-compatible chat API whether they send text or a WAV file. IBM's Speech-to-Decisions demo Space shows the 8B model transcribing audio, writing summaries, screening for harmful content and testing 15 other user-defined flags on one endpoint in real time.

The model card is candid about the limits. The speech encoder and the language model "are not yet jointly trained," with no trained projection coupling them. Audio input is English only, the transcript comes out lowercase and unpunctuated, and there is no decode-time steering. This is a cascade of a speech recognizer feeding a language model inside one server, not a native speech model.

Speed claims and the Doom demo

IBM backs its speed claims with demonstrations rather than a formal benchmark paper. In one, a 30B Granite Switch checkpoint with four specialised aLoRAs raced a Jev-27B setup, described as a frozen Qwen base with one generic decision LoRA, through a five-step retrieval-augmented generation workflow. IBM says its model finished "much faster," without giving a timing table in the post.

A second demo in the repository compares aLoRA and standard LoRA on the same model and hardware. IBM's telemetry screenshot shows the aLoRA server at a 74 percent KV cache hit rate with 5 of 16 queries finished, while the LoRA server sits at 29 percent with 1 of 16 done. IBM publishes a Colab notebook so readers can rerun the race.

The most playful example is Doom. IBM says a custom 3B Granite Switch model made split-second decisions to navigate the game at 60 to 100 decisions per second, against the 10 decisions per second Typesafe reported for Jev, and could narrate and answer questions while playing. That is a vendor demo on IBM's own setup; the post does not say what hardware it used or how well the model played.

How to run or build your own

The quickest route is to serve one of the published checkpoints with vLLM and call it through an OpenAI-compatible client. Switching between the Guardian and Decision adapters is a one-field change on the request, an adapter_name setting in the chat template arguments, while plain chat needs no extra field.

The granite-switch Python package installs with optional extras for composing models, Hugging Face inference, vLLM inference and audio. The README lists Python 3.11 or newer and PyTorch 2.11 or newer, and says the vLLM extra accepts versions from 0.26 to 0.30.

Teams that want a different mix can compose their own checkpoint. The model card shows the exact command used to build the 8B release from the Granite 4.1 8B base, the Guardian and Core adapter libraries, and the speech encoder, and the repository publishes a catalog of 12 adapter functions. IBM recommends its Mellea library for running adapters in applications, though the new Decision adapter has no Mellea shortcut yet. An Adapter SDK for training custom and third-party adapters is planned.

Why it matters and what is still unproven

Most AI products today stitch together several models: one to chat, one to moderate, one to route, one to transcribe. Each adds a server, memory and latency. Granite Switch is an attempt to collapse that stack into one open checkpoint that a company can host itself, and the Apache 2.0 licence makes that practical for commercial use.

The open questions are about evidence. The Decision Index scores, the Guardian comparison with closed decision models, the RAG race and the Doom speeds all come from IBM. The 30B score of 50.1 is respectable for an adapter on a 30B base, but it is not a head-to-head table against the models IBM names, and the blog says more details and data will follow in the coming weeks.

The speech path also needs real-world testing: English-only, unpunctuated transcripts may be fine for screening calls but weak for anything that depends on punctuation or other languages. Until outside benchmarks appear, Granite Switch 4.1 is best read as a well-documented open option for teams that want chat, safety checks and simple decisions on one GPU, with performance claims to verify on their own workloads.