Perplexity's JOLT trains one model to be both teacher and student

What Perplexity published

Perplexity researchers, working with NVIDIA, have described a new way to post-train language models in which one model learns to be both a teacher and a student. The method is called JOLT, short for Joint On-policy Learning and Teaching. It is set out in a paper on arXiv by Randy Ardywibowo and Arnav Dalal of Perplexity and Jiantao Jiao of NVIDIA, first submitted on 7 October 2026, and in a post on Perplexity's research blog titled "A good teacher meets the student where they are". MarkTechPost covered it on 21 October.

The headline claim is about efficiency as well as accuracy. On the MATH benchmark, the authors say JOLT matched the accuracy of GRPO, the reinforcement-learning method widely used to train reasoning models, with 13 times fewer completion tokens and 6.5 times fewer processed training tokens, without giving the student any direct reward. A variant called JOLT+, which adds that reward back, scored highest of all the methods tested in every setting the paper reports.

It is research, not a product. Perplexity has not said whether JOLT is used to train the models behind its answer engine.

The problem with sparse rewards

Reinforcement learning from outcome rewards, the approach behind most recent reasoning models, gives a model a single signal at the end of an attempt: right or wrong, tests passed or failed. On hard, long-horizon tasks, successful attempts are rare and expensive to generate, so the model gets little to learn from. The paper calls this sparse supervision.

On-policy distillation is a popular fix. A stronger teacher model grades each token of the student's own attempt, giving dense feedback along the way. Self-distillation goes further and drops the separate teacher: the same model plays teacher when it is shown privileged information, such as a verified answer, a worked solution, test results or notes from earlier attempts, and plays student when it sees only the task. That approach, known as on-policy self-distillation or OPSD, is cheap because it needs only one model.

Why a smarter teacher can hurt

The paper's central point is that being good at a task is not the same as being good at teaching it. A teacher that has seen the answer may solve the problem through shortcuts the student cannot follow, so its token-by-token guidance does not match what the student would actually do. "Even a higher-performing teacher can provide guidance that degrades student performance," the authors write.

They back this with theory. The paper derives a necessary and sufficient condition for the teacher's distillation update to point in the same direction as the student's own reward gradient, in other words, for the teacher's advice to actually help. The analysis suggests a useful teacher should do well on the task while staying close to the student's current behaviour. In the authors' experiments, disagreement between teacher and student grew over training under OPSD but shrank under joint training.

How JOLT trains both roles

JOLT turns that analysis into a training recipe. A single shared model is trained in both roles at once. The teacher, given the privileged hint, learns from outcome rewards plus a token-level penalty, a Kullback-Leibler term, that keeps its outputs close to what the student would produce. The student, given only the task, learns from the teacher's dense scores on its own attempts. Each role learns from attempts generated in its own context, so both updates stay on-policy, and both change the same weights.

JOLT+ adds the student's own outcome reward to its distillation signal. The Perplexity blog sums up the goal as training the teacher "to meet the student where they are" so the student learns from guidance within its reach.

The idea sits in a busy area. Labs are building better reinforcement-learning infrastructure, such as Alibaba's QwenGyre, and richer training environments, such as FineEnvs, and many of them run on open libraries like Hugging Face's TRL. JOLT targets a different lever: the quality of the feedback inside each training step.

What the experiments show

The paper tests JOLT on five settings with different models and different kinds of hint. On MATH, training the small Qwen3-0.6B with worked solutions as the hint, JOLT reached 72.1 per cent and JOLT+ 72.6, against 68.4 for GRPO, 61.2 for OPSD and 42.2 for another baseline, pi-Distill. On LiveCodeBench v6, training Qwen3-8B on 131 problems with a sibling solution or execution feedback as the hint, JOLT scored 47.6 and JOLT+ 49.1, against 46.1 for GRPO. On GSM8K, using DeepSeek-V2-Lite, JOLT+ beat GRPO by 1.5 points.

The tool-use and agent results are more mixed. On AppWorld, where Qwen3-4B-Instruct learns to operate stateful apps with summaries of earlier attempts as hints, plain JOLT scored 51.9 per cent, below GRPO's 62.2 and pi-Distill's 59.9, while JOLT+ reached 72.1, the best result, and matched GRPO's final training reward in about 1.8 times fewer updates. OPSD collapsed to 29.5.

On Terminal-Bench 2.1, the longest-horizon test, the team trained Qwen3-30B-A3B-Instruct with private verifier information as the hint. JOLT+ led at every measure, from 10.96 per cent at pass@1 to 34.83 per cent at pass@32, against GRPO's 8.25 and 30.34. Plain JOLT beat the base model and both distillation baselines but stayed below GRPO, and OPSD scored below the untrained base model at every level.

Why it matters for training agents

If the efficiency results hold at larger scale, the practical gain is cheaper post-training. Generating long attempts is one of the most expensive parts of reinforcement learning for reasoning and coding models, so needing 13 times fewer completion tokens to reach the same MATH accuracy would matter for any lab running these pipelines.

The approach also suits agent training, where verified answers, test results and logs of earlier attempts are often available during training but not at use time. JOLT offers a way to turn that hindsight into dense feedback the model can use. Perplexity has been publishing more of its model work recently, including open decision models, and the blog suggests a next step: a teacher that also learns to generate training tasks matched to what the student is ready for.

Limits of the evidence

The experiments are small by frontier standards. The largest model trained has 30 billion total parameters with about 3 billion active, and the smallest is 0.6 billion. Results on the Terminal-Bench task come from a single training run per method, and the AppWorld scores have standard deviations of five to six points, so some gaps are within noise.

All results are the authors' own, and the paper has not yet been peer reviewed. The headline token savings are for one benchmark, MATH, measured at matched accuracy, and the authors tuned hyperparameters for each method, which matters because GRPO's results are sensitive to tuning. Plain JOLT, without the student reward, does not beat GRPO on the two agent tasks.

The broader lesson is still useful for anyone building training pipelines: a teacher that knows the answer is not automatically a good teacher. JOLT's bet is that the teacher should be trained against the student's progress, and its results so far suggest the bet pays off when combined with ordinary rewards.