Modal plans NVIDIA Vera CPUs for its agent sandboxes after up to 2x throughput

What Modal announced

Modal, the serverless compute company behind Modal Sandboxes, said on 19 October 2026 that it plans to roll out NVIDIA's Arm-based Vera CPU to power its sandbox product. In a company blog post, engineer Erik Dunteman writes that NVIDIA hand-delivered Modal's first Vera CPU server to its San Francisco office the week before, and that the machine is now running in a rack there.

The headline claim is Modal's own: in early runs of a benchmark it built for agent-style workloads, Vera delivered up to double the task throughput of an equivalent x86 machine based on Intel's Granite Rapids (GNR) generation, while individual tasks finished in the same time or faster. Modal says that result makes Vera "an incredibly compelling option" for Sandboxes and that it will work with NVIDIA and cloud providers to scale up a Vera fleet.

It is a notable shift for Modal. The post says the company has a long history of running x86 machines exclusively, and that supporting Arm will require changes across its infrastructure and image build system. Reuters headlined the news as NVIDIA winning Modal's CPU business, a market Intel and AMD x86 chips have long dominated.

Why sandboxes stress CPUs differently

Modal Sandboxes are isolated containers and virtual machines that AI agents and training jobs use to run code. Modal says orchestrating them means packing hundreds of concurrent containers and VMs onto each cloud host, and that the resulting clusters are noisy in ways ordinary server fleets are not.

Usage is spiky. A sandbox can sit mostly idle while it waits on a language model, then jump to ten times its baseline CPU use for an npm install or a heavy grep. The shape of the work is only known at runtime, because a model can throw anything into the box. One tenant's burst can slow another tenant on the same host, the classic noisy-neighbour problem.

Modal lets users reserve capacity for their typical, median usage rather than their worst-case spike, and relies on spare headroom in each host to absorb bursts. The company says this has saved users millions of dollars, but it also increases exposure to contention. That makes a CPU that stays fast when crowded unusually valuable to this kind of business.

Modal names two main customers for that speed. Background agents, such as Ramp Inspect and agents running on Niteshift, earn more when they answer faster. Reinforcement-learning rollouts leave GPUs idle while sandboxed code runs, because most training loops still wait for every rollout to finish. Modal has written before about squeezing that pipeline, including mixed-precision RL samplers across six GPU types; in its words, any time saved on CPU is time saved on GPU.

How Modal benchmarked Vera

To test Vera, Modal built what it calls Agentless Agent Bench and published it as an open repository. It takes the terminal-bench 2.1 task set, uses the oracle solutions that skip model turns, and filters them to 50 tasks that need no network. The Dockerfiles are ported to run on Arm, and environments are built and pre-warmed before a run so that network time does not leak into the results.

Every run executes the task's own terminal-bench verifier right after the oracle solution, and a run only counts if the verifier passes. The tasks cover what agents and RL jobs actually do on a CPU: compiling code, searching files, git operations, device emulation and similar chores.

The repository offers three modes. A serial mode runs each task one at a time and records per-task latency. A throughput mode runs all 50 tasks concurrently in a fixed window, counting completed solve-and-verify cycles. A ramp mode sweeps the throughput test from one core up to all cores and plots completed runs against core count.

Latency on a quiet machine versus throughput under load

Dunteman writes that the serial results first made him sceptical. On an unloaded machine, across the 50 tasks, Vera often beat the other CPUs Modal tested, but not always and not by a large margin.

The throughput runs told a different story. Normally, packing more work onto a host raises throughput until the CPU saturates, after which each task slows down. Modal says Vera did not follow that curve: it kept finishing tasks at its original pace as jobs piled up, so throughput climbed while per-task latency stayed equal to, or sometimes lower than, the x86 machine. Modal also says the alternatives hit throughput plateaus early as CPU counts rose, while Vera kept improving, suggesting the company could push it further than the single node it had.

Modal attributes the difference to memory bandwidth, which it says lets far more container workloads share each core without throttling. The post reports that when it asked, NVIDIA's Vera team said the results were not surprising.

What Vera is

NVIDIA launched Vera at GTC on 16 March 2026 as a CPU built for agentic AI and reinforcement learning. The chip has 88 custom NVIDIA-designed Olympus cores, each able to run two tasks using what NVIDIA calls Spatial Multithreading, and a second-generation LPDDR5X memory subsystem that NVIDIA rates at up to 1.2 TB/s, which it says is twice the bandwidth at half the power of general-purpose CPUs.

Vera is also the host CPU in NVIDIA's Vera Rubin NVL72 systems, where it links to GPUs over NVLink-C2C, and in reference designs for HGX Rubin NVL8. NVIDIA has been publishing rack and data-centre specs around that platform, including Vera Rubin MGX designs at the Open Compute Project. At launch it also described a standalone rack of 256 liquid-cooled Vera CPUs that it said could sustain more than 22,500 concurrent CPU environments.

In March, NVIDIA said Vera was in full production for partner availability in the second half of 2026, and listed cloud providers planning to deploy it, including Alibaba Cloud, ByteDance, Cloudflare, CoreWeave, Crusoe, Lambda, Nebius, Nscale, Oracle Cloud Infrastructure, Together AI and Vultr. Its launch release also quoted Cursor's chief executive on using Vera for coding-agent workloads.

What changes for Modal and its users

For Modal's scheduler, the company says Vera means higher packing density and more forgiving burst protection, because a crowded host does not punish each task. For Sandbox users, Modal says it can offer higher absolute scale and throughput without giving up latency.

The post does not give a launch date, pricing, regions or instance shapes for Vera-backed Sandboxes, and it does not say what share of the fleet will move. Users should also expect the usual Arm caveats: container images and native dependencies built only for x86 will need Arm builds, which is exactly the image-system work Modal says it has to do on its side.

The move puts Modal among companies betting that agent workloads are as much a CPU problem as a GPU one. AMD has pitched the same idea with its Helion rack blueprint for running agents on EPYC CPUs, and agent builders have been releasing their own isolation layers, such as Perplexity's open-source SPACE Firecracker sandbox.

Limits and open questions

Every number here is Modal's, from early runs on a single Vera node, and the post presents results mostly as charts without a full table of figures. AI Lookout has not run the benchmark. Modal says up to double the throughput, which describes the best case in its tests rather than a typical gain across all workloads.

The comparison set matters too. The headline ratio is against an equivalent Intel GNR machine; Modal's charts also include other CPUs, and the post does not claim Vera wins every task. The benchmark strips out network calls and model turns by design, which isolates CPU behaviour but leaves out the time agents spend waiting on models and the network in real use. The repository had no licence file when AI Lookout checked, so reuse terms for the benchmark code are unclear.

Finally, NVIDIA's own efficiency and speed claims for Vera come from its launch materials. Independent results from other operators, on public instances, will be the real test.

Why it matters

Agent products are driving a surge in short, bursty CPU jobs, from running tests to compiling code to replaying RL episodes. Modal's post is one of the first detailed public accounts from a sandbox operator of how a specific CPU behaves under that mix, and its claim is specific: Vera's advantage shows up when the machine is crowded, not when it is idle.

If that holds outside Modal's lab, it gives NVIDIA a foothold in the CPU side of AI clouds, a market where Intel and AMD x86 chips have been the default. It also gives other sandbox and agent platforms an open benchmark to rerun on their own hardware, which may matter more than any single vendor ratio.