Skywork open-sources Matrix-Game 4.0, which renders playable low-poly games in real time

What did Skywork release on 15 October?

Skywork AI’s Matrix-Game repository added a fourth generation this week. Its README dates the release to 15 October 2026 and says inference code, a browser demo and the full game source are available, with model weights and game assets for three scenes—Medieval Castle, Ancient Egypt and City Racing—hosted on Hugging Face. The commits that publish the code are timestamped 14 October in UTC, and a further commit adding RTX 5090 inference landed early on 15 October.

Matrix-Game has been Skywork’s line of open interactive world models since May 2025. Version 1.0 arrived on 12 May 2025, version 2.0 on 12 August 2025 for real-time long video generation, and version 3.0 on 27 March 2026 with long-horizon memory. Version 4.0 changes the recipe more than the scale: rather than generating a world from pixels alone, it pairs a conventional, executable game with a video model whose only job is to make that game look finished.

Skywork’s tagline on the project page is “Code builds the game, AI renders the world in real time.” That page is still labelled a preview, and its Paper and Code buttons say “soon,” even though the code is already public on GitHub. A technical paper had not appeared when we checked.

How does a game engine plus a video model make a playable game?

The game engine keeps the world state, applies game rules and physics, and reads the player’s keyboard and mouse. It then draws a deliberately cheap low-poly proxy image of each moment. The video model reads that proxy stream and repaints it, chunk by chunk, into detailed frames. Heads-up display and sound stay outside the model as layers aligned to each presented frame. The design is close in spirit to Microsoft Research’s Code Video Model, which also has code render a rough proxy before a video model makes it realistic, but Matrix-Game does it continuously while someone plays.

Skywork argues that world models which only roll out pixels keep the world state implicit: once something leaves the view it is gone, and game rules have to be re-inferred from every frame. Wrapping the model in what the project page calls a harness avoids that. Whatever the rules can decide stays in the engine, so an enemy behind a wall still exists and a lap timer still counts.

Each scene checkpoint has 5,001,016,512 parameters in BF16 and is fine-tuned from Alibaba PAI’s Wan2.2-Fun-5B-Control. According to the architecture notes, a chunk is one latent frame that decodes to four RGB frames at 864×480. Generation is causal: each new chunk attends to a bounded cache of recent chunks and a fixed reference, or “sink,” image that anchors the scene’s look. The denoiser is distilled to one or two steps, which is what makes streaming plausible on one GPU.

The project page describes more ambitious pieces: a coding agent that turns a player’s request into new engine rules, prompts written from engine state, multi-reference appearance and spatial memory retrieved by camera pose. The released shooter configuration is narrower. Its documentation lists a fixed scene prompt and “no automatic cache reset, SDEdit initialization, retrieved memory or dynamic prompt generation.” City Racing does support dynamic captions supplied by the engine, at a throughput cost.

How fast is it, and on which GPUs?

Skywork publishes two kinds of numbers, and they measure different things. The v4 README gives reference measurements on one NVIDIA A100-SXM4-80GB at 864×480 with one denoising step, the lightweight TAEHV codec and CUDA graphs: about 38–39 FPS of model processing for Castle and Ancient Egypt, about 38.5 FPS of server-side pipeline throughput for Racing with a fixed prompt, and about 28.9 FPS with dynamic prompts. The racing figures come from an earlier checkpoint with the same architecture.

The README is explicit that none of those is browser presentation frame rate or action-to-photon latency. The “--fps 40” setting in the demo is described as a target capture cadence, “not a throughput or latency guarantee,” because video encoding, network transport, buffering and browser decoding all add delay.

The project page headlines faster consumer-GPU figures: 83 FPS with one-step denoising on an RTX 5090, 48 milliseconds to denoise one chunk, 112 milliseconds from a key press to the first frame that reacts, and the text encoder, diffusion transformer and VAE all resident on a single card. The matching “rtx5090” preset swaps in FP8 linear layers, fused kernels and CUDA graphs; a second preset adds a patched SageAttention build. For context on how fast neural rendering is already moving in shipping games, see NVIDIA’s lighter DLSS 5 neural rendering model, which does a much narrower job than repainting every frame.

What can you download and run today?

Everything needed to play is split between GitHub and Hugging Face. The model side needs Linux, an NVIDIA GPU and Python 3.12; the release was tested on an A100 80 GB with PyTorch 2.12.0 and CUDA 13.2. The standalone games build on macOS or Linux with CMake, C++20, SDL2 and FFmpeg, and they run without the neural renderer at all, which is useful for testing gameplay or recording input.

A downloader fetches one scene and its shared dependencies from a pinned Hugging Face revision and checks every file’s size and SHA-256 against a manifest. The shared umT5 text encoder, tokenizer and Wan VAE are pulled from the official Wan2.2 repository rather than duplicated. The weights only work with Skywork’s inference code; the model card says they are not drop-in Diffusers pipelines.

Two presets trade speed for fidelity. “Realtime” uses TAEHV encoding and decoding, a single denoising step and the accelerated runtime; “quality” uses the full Wan VAE and two steps and is the default when no preset is named. Offline mode turns a recorded low-poly video into a rendered MP4. Online mode runs a single-user WebRTC demo through MediaMTX that you open in Chrome or Safari; the client needs no GPU.

The README also invites people to customise the games with a coding agent, editing terrain, object placement, NPC behaviour and rules, then testing changes locally without loading model weights. Licensing is permissive: Skywork lists the v4 code, scene weights, reference images and game assets as Apache-2.0. TAEHV is MIT-licensed, and some racing audio is CC0 or CC BY 3.0 with credits that must be kept.

How does Matrix-Game 4.0 compare with other world models?

Most recent world models try to generate the world itself. Noiz AI and HKUST’s Helix World, released three days earlier, turns one image into a scene you steer with WASD and generates matching stereo sound at 24 fps on one H800. Odyssey’s Odyssey-3 is a closed foundation world model shown through a research preview. In both, the model is responsible for what happens as well as how it looks.

Matrix-Game 4.0 gives up some of that generality on purpose. It cannot invent a new game from a sentence, and each released checkpoint is specialised to one scene. In exchange, collisions, scores and enemy positions follow the engine’s deterministic rules, and the model cannot forget an object that walked off-screen. For studios, that division of labour may be the more practical path: keep the gameplay code they already trust and treat the model as a learned art pass.

Shipping three playable games rather than video clips also makes the claims easier to check. Anyone with a suitable GPU can play the same scenes, record the low-poly input and compare it with the rendered output, which is harder to do with a closed preview.

What the release does not include yet

Training code and training data are not included, so outside researchers can run and fine-tune around the released checkpoints but cannot reproduce how they were made. The coding-agent harness, retrieved spatial memory and dynamic prompting shown on the project page are not part of the released shooter configuration, and the page previews four games while three ship.

The browser demo is described as a trusted, single-user setup rather than a cloud-gaming service: HTTP, RTSP and WebRTC bind to loopback by default, and the README warns not to expose them to the internet. One server process supports one active game session and one scene model, and each session has a finite positional budget—8,192 generated chunks by default—after which a new session is required.

Finally, every performance number is Skywork’s. We read the repository, model card and project page and inspected the published reference images, but we did not install the models, play the games or measure latency. Independent tests on RTX 50-series cards will show how close the 83 FPS and 112 ms claims come in a real browser session.