ZJU open-sources SuperNav, an agent harness that navigates without fine-tuning the MLLM

What did Zhejiang University publish?

Researchers led by Sida Peng at Zhejiang University’s 3DV group posted SuperNav: An Agentic Navigation System for Any Task in Any Scene to arXiv on 8 October 2026 (cs.RO / cs.CV, 2610.12126v1). The same day they published the project page and opened github.com/zju3dv/SuperNav. Co-authors span Zhejiang University, Shenzhen University and Causa Robotics.

The pitch is task and scene generality without navigation-specific fine-tuning of the multimodal model. End-to-end vision-language-action navigators, the authors argue, inherit the coverage of their navigation trajectories. Modular pipelines hard-code when the model may explore, confirm, or stop. SuperNav instead treats the MLLM as a decision-maker that calls tools, reads optional Markdown “Navigation Skills,” and keeps a textual task-progress ledger across steps.

That framing sits next to other October robot-learning drops we covered, including Arc’s action-grounded robot traces and DreamTrue’s robot world-model preprint, which also emphasise scaffolds around pretrained models rather than training a new policy from scratch.

How does the agent harness actually work?

At each step the model receives four direction-labelled RGB views (front, right, back, left). To move, it calls a visual-point tool with a view id and normalised image coordinates. The tool executes motion and returns fresh views plus short diagnostics. The agent may also turn in place, update pending/found goals, load a Skill, or close the session as achieved or blocked.

Two motion backends share the same point interface. The Geo-based Executor back-projects the selected pixel with depth and follows a path on a navigation mesh (or LiDAR height map on the Unitree Go2). The Learned Executor is a NoMaD-inspired flow policy that predicts local displacement increments and a stop signal from a marked reference image and a short RGB history, without depth grounding. Ablations in the paper credit Navigation Skills and the four-view tool design for large drops in success when removed.

The agent loop is implemented with Codex CLI and Model Context Protocol tools. Bundled recipes default to gpt-6-astra with medium reasoning; an ablation swapping in GPT-5.6 Terra still runs the same harness. Skills are Markdown packages the model discovers by name—search, recovery, entrance finding, completion checks—rather than hard-coded finite-state machines.

What do the author-reported benchmarks show?

On a new InteriorGS-based Habitat-GS benchmark (150 single-object and 150 ordered multi-object tasks), SuperNav + Geo-based Executor reaches 78.00% SR / 0.4127 SPL on single-object goals and 34.00% SR on multi-object goals. UniNaVid is the strongest single-object baseline at 34.00% SR; OmniNav’s Action Former leads the multi-object baselines at 4.00% SR. Demand-driven navigation on 200 adapted Demand-Bench tasks in AI2-THOR shows 59.50% SR for SuperNav versus 37.50% for OmniNav Action Former.

Category-level HM3D-OVON and HM3Dv2 tables mix the authors’ 120-episode OVON subset and 1,000 HM3Dv2 episodes with published numbers from MTU3D, SoftNav, AstraNav-Memory, ApexNav and others. SuperNav’s Geo-based Executor posts 73.33% SR at 1 m on their OVON subset and 86.50% SR at 1 m on HM3Dv2. The paper warns that evaluation subsets, stop rules and action budgets differ across systems, so those columns are not a single controlled bake-off.

Real-world demos on a Unitree Go2 with four-view RGB and LiDAR-based geometric execution appear on the project page—printer then trash bin, basketball search, finding another Go2. The repository README still checks those drivers as unfinished for download.

What can you download today, and under what licence?

The GitHub tree includes the agent runtime, tools, Skills, experiment configs, tests and evaluation utilities for Habitat-GS and AI2-THOR. Installation targets Python 3.12 with optional extras for agents, evaluation, AI2-THOR and a Rerun viewer. Learned Executor weights are on Hugging Face under the0xka1/SuperNav-Learned-Executor (created 9 October, last modified 10 October 2026). The README still marks the InteriorGS single/multi-object packs, the AI2-THOR demand-driven pack, and real-robot Go2 drivers as not yet released.

The licence is Project Registration License (PRL) v1.0, not Apache or MIT. Academic, educational, evaluation and personal use are free without registration. Any company or organisation that uses the software in an internal or external project must register that project through a Google Form before use; registration is free and said to grant permission automatically. That is a material difference from permissive OSS for product teams planning a fork.

We did not run the Habitat stack or the Go2 stack in this newsroom. Claims about live robot behaviour rest on the authors’ videos and paper text.

How does this sit next to other robot navigation work?

October’s robotics feed already includes world-model and policy-scaffold work such as Meta FAIR’s RoboJEPA latent world models, NVIDIA’s Long-WAM world-action context, and test-time helpers like SpatialHarness test-time scaffolding. SuperNav’s distinctive claim is organisational: keep the frontier MLLM’s general competence, give it robot-shaped tools and skills, and swap motion backends without retraining the reasoner.

That is closer in spirit to agent harness engineering on desktops than to classical ObjectNav fine-tunes. Teams already wiring MCP servers around Codex or Claude for software tasks will recognise the pattern; the physical twist is the visual-point contract and the LiDAR or mesh follower behind it. For industrial context, see also the AWS Physical AI toolchain.

Limits are explicit. Semantic success still depends on the underlying MLLM. Geometric execution needs maps or reliable depth. Inference latency makes episode duration variable. Demand-driven scoring measures ordered arrivals at relevant objects, not full activity completion. Long multi-stage demand sequences remain hard—no method in their table finishes six or more stages.

What did we not verify, and what should teams watch?

We inspected the arXiv abstract and HTML, the GitHub README and LICENSE, the project page, and the Hugging Face model card metadata on 12 October 2026. We did not re-run Habitat-GS, AI2-THOR, HM3D, or the Unitree demos. Baseline checkpoints and camera settings follow the paper’s Appendix; small protocol mismatches can move SR by more than a few points.

Watch for the missing evaluation packs and Go2 drivers, any third-party reproductions on public HM3D splits, and whether organisational users actually register under PRL. If the authors later open an Apache-licensed subset of the tools, that would change the adoption story. Until then, treat the headline numbers as author-reported on author-defined suites, and the open code as a research harness with a registration gate for company projects.