Announced 16 Oct 2026 · Sources checked
What Microsoft and NVIDIA published
Microsoft and NVIDIA published a joint paper on 16 October 2026 called Risk Management for Open-Weight Models in Enterprise. It is a 22-page guide for companies, governments and other organisations that download model weights and run them on their own infrastructure, and it is billed as the first in a planned series.
Michael Scovetta, a partner security assurance manager at Microsoft, introduced it in a Microsoft Security blog post. The paper was edited by Andrew Berkley and April Chen of Microsoft and Michael Boone, Eileen Long and Arun Rao of NVIDIA. Listed contributors include Natasha Crampton of Microsoft and Rich Harang and Daniel Rohrer of NVIDIA.
Alongside the PDF the companies opened a companion GitHub repository in NVIDIA's organisation. It holds the paper, the risk taxonomy, a table mapping evaluations to risks, and an empty folder for mitigations. Changes to the taxonomy need sign-off from a maintainer at each company.
The core claim: openness alone does not set the risk
The paper's central sentence is repeated in its abstract, summary and conclusion: weight openness alone does not determine an AI system's risk. Risk depends on the model, the use case, the deployment and the safeguards. Open and closed models share most underlying risks, the authors argue, but differ in who has to manage them.
Closed models usually arrive behind an API with guardrails, security monitoring and other services already attached. Open-weight models are often distributed as bare checkpoints. The controls needed are largely the same, the paper says, but the enterprise running the weights may have to supply them itself. Model access and deployment are separate choices, and open models can also be used through managed platforms.
The framing matters because open weights are now a large part of the market. In the past week alone, Meta released Muse Spark Open and NVIDIA shipped Nemotron 3.5 Super VL and Bolt, both as downloadable weights.
Five categories and 30 risk types
The taxonomy sorts risks into five top-level categories: Cybersecurity, Data Protection, Content, Human-AI Interaction and Bias. Under them sit 30 more specific risk types. The authors say the five categories are meant to stay stable while the risk types are updated as new risks appear, and that owners, evaluations and mitigations should be assigned at the level of the risk type.
Some entries are familiar, such as hate and violence, child safety, PII disclosure or CBRNE weapons uplift. Others reflect how models are used in 2026. Cybersecurity includes malicious tool calls by agents, extraction of a deployment's safety policy, runaway token consumption and loss of control, defined as an operator losing the ability to observe, correct or shut down a running system. Human-AI interaction lists sycophancy, dark patterns, anthropomorphism, overreliance and inappropriate relationships.
Data Protection includes model theft and model distillation, defined as systematic querying to harvest a model's outputs for training a student model, typically in breach of its terms. The paper deliberately leaves out attack techniques such as jailbreaks, prompt injection and adversarial suffixes, treating them as routes to a risk rather than risks in themselves, and it excludes society-wide harms that no single company manages.
How the paper says to assess a model
For process, the paper does not invent a new framework. It applies the five steps of ISO/IEC 23894, the AI version of the ISO 31000 risk standard, and lines them up with the Map, Measure and Manage functions of the NIST AI Risk Management Framework: define scope and acceptance criteria, assess risks, run evaluations, mitigate, and monitor after deployment.
Several points are sharper than usual for vendor guidance. The authors warn that safety behaviour is often more fragile than capability, so fine-tuning, merging, quantization or repackaging can strip safeguards while leaving task performance intact. Safety and capability should be tested as separate questions.
They also separate propensity from likelihood. An evaluation measures how often a model shows a behaviour under test; likelihood is what that becomes once real users, exposure and motivated attackers are added. Reporting the first as the second, the paper says, overstates what an evaluation proves. Threat modelling is recommended where a risk needs an adversary, and failure modes and effects analysis where it does not.
Provenance before behaviour
Because an open model may be modified or repackaged before an enterprise receives it, the paper tells teams to establish provenance before testing behaviour. It names OMS Model Signing as a way to verify that the artifact received is the one the provider published, along with the claims in its model card.
The wording is blunt: no volume of behavioural testing establishes what an artifact is, only how it behaves. Where signatures are unavailable, the paper says, the unverified integrity should be recorded as an open risk rather than treated as closed by good evaluation results.
That echoes a wider concern about treating models as part of the attack surface, an idea Microsoft chief executive Satya Nadella raised when he described models as potential insider risks. Monitoring guidance in the paper follows the same logic: watch for new derivatives, licence or repository changes, new jailbreaks and runtime vulnerabilities, and keep fast-updating filters rather than waiting for retrained weights.
The evaluations it points to
An appendix maps a selection of risks to public evaluations. It lists CWEval for insecure or malicious code, Google DeepMind's dangerous-capabilities suite for loss of control, CIMemories for PII disclosure, MLCommons AILuminate for copyright and harmful content, NVIDIA's Nemotron Content Safety Dataset v2 and WildGuard for hate and sexual content, and AA Omniscience, LongFact, HalluHard and GaRaGe for groundedness.
For bias it points to a modified BBQ benchmark from NVIDIA, and for sycophancy to MASK. The authors stress that adversarial testing and expert red teaming are not interchangeable: the first is automated and repeatable, the second is open-ended human exploration, and high-severity risks warrant both plus independent evaluation.
Each company also points to its own tools. The paper mentions Microsoft Foundry's built-in evaluators, automated red teaming and content-safety filters, and NVIDIA's NeMo Gym for repeatable evaluation. Both mentions come with caveats to check what a given deployment actually supports, including preview status.
A worked example: one endpoint, two layers
The paper closes with one worked case: an industrial equipment manufacturer hosting an open-weight model as a plain text endpoint. Support engineers use it to summarise field reports, translate manuals across the 14 languages of its dealer network and classify inquiries, and the same endpoint is offered to independent dealers under contract.
Because the service has no retrieval, tools or memory, the authors say controls can sit in only two places: the model, which should be evaluated directly on the company's own documents in all 14 languages, and the serving layer, with authentication, rate limits, abuse monitoring and content filtering. External dealer traffic should get tighter limits than internal use.
Two details stand out. The paper warns that a fabricated torque specification reads exactly like a correct one, which is why groundedness is a top risk for a service with nothing to ground against. And it notes that filtering is often done by a second model, so a deployer ends up with two models to evaluate.
What is missing and what comes next
The first paper covers only the model layer. Guidance for the runtime, the agent harness and the application, where tool permissions and data access create most agent risk, is promised for later papers, and the repository's mitigations folder is still empty. The evaluation table covers a selection of risk types, not all 30.
The repository's licence file currently reads only "To Be Determined for Future Evals", so reuse terms for the taxonomy and table are unclear. The paper is also a framework rather than a test: it reports no measurements of any model. Readers wanting the broader policy picture can compare it with the UK government's frontier AI risk package, which takes a regulator's view of similar ground.
For security and risk teams, the useful part is the shared vocabulary and the insistence on testing the exact artifact and configuration they deploy, rather than relying on a model's reputation or its open or closed label.
Common questions
What are the five risk categories?
Cybersecurity, Data Protection, Content, Human-AI Interaction and Bias. Together they contain 30 risk types, with owners and evaluations meant to be assigned at the risk-type level.
Does the paper say open-weight models are riskier than closed ones?
No. It argues weight openness alone does not determine risk; the use case, deployment and safeguards do. What changes is who must supply controls that a closed API would otherwise provide.
Where can I get it?
The PDF and taxonomy are in the NVIDIA/OpenEnterpriseRiskManagementGuidance repository on GitHub, linked from Microsoft's 16 October Security blog post. The repository's licence is still listed as to be determined.
Sources & further reading
How this story was made
Written by Kristian Kostov with AI assistance and checked against the linked sources. Company performance claims are attributed to the company. Analysis reflects AiLookout’s interpretation; we have not independently tested the products discussed. The cover places the companies’ own logos, taken from public-domain or openly licensed files credited in the caption, on a plain background. It is not a product photograph, a screenshot, or an endorsement.
Our editorial standards
