gpt-oss-120bbenchmark results

Measured speed and quality

Speed (single user)

Source: fleet-bench/2026-09-23-dgx1-moe-gpt-oss-120b-mxfp4.json, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-09-21-h200monsta-gpt-oss-120b-judge-shape.json, ai11/RESULTS-2026-10-13-model-sweep.md

Aggregate throughput

Source: fleet-bench/2026-09-23-dgx1-moe-gpt-oss-120b-mxfp4.json, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-09-21-h200monsta-gpt-oss-120b-judge-shape.json

Prefill and latency

Source: fleet-bench/2026-10-02-best-model-per-tier-results.md

Context

Source: fleet-bench/2026-10-02-best-model-per-tier-results.md

Quality

Source: fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-10-03-cmmc-rag-v3-results.md

Power and efficiency

Source: fleet-bench/2026-10-02-best-model-per-tier-results.md

Other measurements

Source: fleet-bench/2026-10-03-cmmc-rag-v3-results.md, fleet-bench/2026-09-21-h200monsta-gpt-oss-120b-judge-shape.json, fleet-bench/2026-10-02-best-model-per-tier-results.md, ai11/RESULTS-2026-10-13-model-sweep.md, fleet-bench/2026-09-27-strix-vs-gb10/README.md

# gpt-oss-120b

What it is

gpt-oss-120b is an open-weight mixture-of-experts model with about 117B total parameters, of which roughly 5.1B are active on each token. It ships in its native MXFP4 quantization, which is how we ran it on every configuration in this report: on a single GB10 (DGX Spark class), on the DGX Station GB300 (dgx1), on H200 hardware, and on Ryzen AI Max+ laptops in the Strix Halo class. Because only a small slice of the network fires per token, it decodes far faster than a dense model of the same total size, but it still needs more than 60 GB of memory just for the weights, which rules out anything smaller than a one-hundred-twenty-eight-GB-class unified-memory machine or a discrete GPU server.

The model is a reasoning model: it produces thinking before the answer, and on some engines thinking cannot be switched off. That matters when you compare it against other models on this site, and we call out every place the measurement conditions differ. Its strongest measured trait on our fleet is clean tool hygiene: across the 27-case tool-call battery it produced zero hallucinated tool calls, and in the grounded compliance battery it fabricated at most a single citation per run. If you need a model that rarely invents a tool call or a citation, this one has the best record we measured.

The trade-off is speed and capacity. On a single GB10 it is the slowest mixture-of-experts model we tested, and its context window is capped at its model limit, roughly 131k tokens, half or less of what several competitors serve. It is a quality-per-token pick, not a speed pick, and the sections below show exactly where that trade lands on each hardware configuration.

Where it runs best

The clear winner is the DGX Station GB300. On dgx1 with vLLM, one user measured 390.7 tok/s and 32 concurrent users reached an aggregate of 4,924.2 tok/s. That is roughly eleven times the single-user speed of the same model on a GB10 Spark and more than twice the ceiling aggregate of the four-way H200 judge server, from a single workstation. If gpt-oss-120b is the model you want and you can afford the hardware class, run it there.

On H200 hardware the model behaves like a service engine rather than a desktop engine. On one H200 GPU (h200monsta) under our SEO-judge workload it decoded 179.0 tok/s aggregate for one user, with a per-stream median of 203.5 tok/s and a first-token time of 0.904 s on a 3,098-token prompt. Aggregate throughput climbed to 570.0 tok/s at 8 users and 1,118.1 tok/s at 32 users, peaking at 1,511.2 tok/s with 64 concurrent users, the measured ceiling. First-token latency at 32 users rose to 7.99 s, so plan queue capacity rather than expecting instant responses at high concurrency.

On a single GB10 (DGX Spark class, vLLM nightly, MXFP4) one user gets 35.1 tok/s on prose and 35.0 tok/s on code. That is usable for one person, but it is the slowest MoE model in our GB10 tier, and the concurrency ceiling is low: the engine held the 10 tok/s per-user floor only up to 16 users, with aggregate decode of 198.0 tok/s at that ceiling, against 290.5 tok/s at 32 users measured past the comfortable per-user floor. Prefill is also modest at 4,900 tok/s for an 8k prompt and 4,037 tok/s at 32k, and the 128k prefill test did not run because the model's served context stops at that same model limit. Energy efficiency follows the same pattern: 1.63 tok/J at ceiling, several times behind the small MoE models. On the MSI variant of the same GB10 hardware the numbers matched within a couple of percent, so either build performs the same.

On Strix Halo laptops the model fits and runs, with one serious caveat on Windows. On ai11 (a large unified-memory pool) with llama.cpp Vulkan it measured 41 tok/s while scoring 34 of 40 on MMLU-Pro and 19 of 20 on HumanEval. On the g1a Windows laptop we measured both backends: ROCm held 40.4 tok/s median generation and still delivered 32.3 tok/s at 32k context, while Vulkan collapsed at 32k to single-digit tok/s. Prompt processing on ROCm ran at 685 tok/s at short context, 587 tok/s at 8k and 359 tok/s at 32k. If you serve this model on a Strix Halo Windows machine, use the ROCm build; the Vulkan path cannot keep the model and long-context KV cache resident.

Measured speed

Single-user decode by hardware (gpt-oss-120b, MXFP4):

• DGX Station GB300 (dgx1), vLLM: 390.7 tok/s. Measured with the reasoning-aware harness variant, so thinking is on and this row is not comparable with thinking-off rows.
• One H200 GPU (h200monsta), vLLM, thinking off: 179.0 tok/s aggregate, 203.5 tok/s per stream, first token in 0.904 s.
• One GB10, vLLM nightly: 35.1 tok/s prose, 35.0 tok/s code.
• ai11 (Strix Halo, the large-memory Minisforum box), llama.cpp Vulkan: 41 tok/s, default reasoning effort.
• g1a (Strix Halo, Windows), Ollama ROCm: 40.4 tok/s median across the four bench cases; 44.7 tok/s on the Vulkan short-context run, but only single-digit tok/s at 32k context, which is why we recommend ROCm there.

Aggregate throughput under load:

• GB300: 4,924.2 tok/s at 32 users.
• H200: 570.0 tok/s at 8 users, 1,118.1 tok/s at 32 users, 1,511.2 tok/s at the 64-user ceiling.
• GB10: 290.5 tok/s at 32 users, with the per-user floor reached at 16 users where aggregate was 198.0 tok/s.

Prompt processing on one GB10 ran at 4,900 tok/s for 8k prompts and 4,037 tok/s at 32k. That is fast enough that a 30k-token document adds only a few seconds before the first token. The 128k point was not measured because the served context is capped at the model's own limit.

Context: we verified a needle at 90 percent depth of the served window on a single GB10. A needle at 120,755 tokens passed in 73 s, and single-user decode at a prompt just short of the context ceiling still produced readable output, though slower than at short context. The model's native context limit is 131k tokens, its model limit, and vLLM on one GB10 could not serve more than that. Several competing models on this site serve 262k or more; if your documents routinely exceed the low 100k range, pick one of those instead.

Quality and accuracy

On the GB10 quality battery, gpt-oss-120b scored 10 of 10 on the code checks and 22 of 26 on long-context items, with 29 of 38 on the tools battery and 11 of 15 on the agent battery. The headline is tool hygiene: zero hallucinated tool calls in the 27-case tool-call battery, a distinction no other GB10 model in our tier matched, and an agent completion rate with minimal fabrication. If your workflow chains many tool calls, unreliability compounds, and this model measured cleanest.

The grounded compliance battery is where we would direct a compliance team. On one GB10 against the CMMC RAG v3 API it scored 77 strict of 93 on the agentic ladder (seed 21, with the other seeds cut by an infrastructure outage), 82 correct, one fabrication, all 10 trap items caught, 8 of 10 on the work tasks, and nearly every tool call it made was valid. On the code-first and frozen-passage variants it scored 0.889 and 0.86 strict respectively, averaged over three seeds, with 0.760 on the overall strict measure. Its closed-book strict score was 0.333, which is the expected pattern: it needs the grounded index, not memory. A separate reference run on four H200s in FP8 scored 61 agentic strict, so treat the exact strict number as sensitive to hardware, quantization and harness; the stable finding is that grounding lifts it well clear of its closed-book floor.

On the small 60-item quality battery on ai11 it scored 34 of 40 on MMLU-Pro and 19 of 20 on HumanEval at default reasoning effort, measured at 41 tok/s. That is a small sample; use it to rank, not as a certificate. The weakness to plan around is budget: as a reasoning model it spends tokens thinking, and our long-context harness showed slower effective throughput on near-maximum-context prompts. Give it a generous output budget in your serving configuration, or its reasoning can crowd out the answer.

How we ran it

Every number on this page comes from our own measured runs, stored with their source files, on the Petronella Technology Group, Inc. fleet.

• One GB10 (DGX Spark class, spark and MSI builds) and the four-unit H200 reference used vLLM (nightly on GB10), MXFP4, thinking off where the engine allows it, temperature 0 for speed runs, closed-loop concurrency ladders from 1 to 32 users, three shuffled repetitions, 512-token outputs. The GB10 pod ran with all units at locked clocks; the MSI copy matched the Spark within about 2 percent on every point, so we report them as one configuration.
• The GB300 run on dgx1 used vLLM with the reasoning-aware harness variant; thinking cannot be disabled on gpt-oss, so that row is not comparable with thinking-off rows and we do not mix it into cross-hardware speed claims.
• The H200 concurrency ladder used the live SEO-judge workload shape: a 3,098-token prompt with a fixed output cap, thinking off, one GPU.
• Strix Halo runs used llama.cpp (Vulkan and ROCm builds) on ai11 under Ubuntu and on g1a under Windows 11, MXFP4 GGUF, with the standard llama-bench short and long-context points, plus our 60-item quality battery at temperature 0.
• Quality batteries: the GB10 battery covers tools, agent, code, writing and long-context items; the grounded compliance battery runs the model against a read-only CMMC RAG API with strict grading that requires a valid supporting citation, multiple seeds where noted, and fabrications counted.
• Failures and caveats we did not hide: the Vulkan collapse at 32k context on Windows is reproduced across runs and is a real constraint, not a fluke; one H200 reference run in FP8 scored lower on the agentic ladder than the GB10 MXFP4 run, and we report both rather than cherry-picking; and two of the three GB10 agentic seeds were cut by an outage, so the 77 strict figure rests on one seed.

When to pick it

Pick gpt-oss-120b when citation and tool discipline matter more than raw speed. It had the cleanest tool-call record of any model we measured on the GB10 tier, with zero hallucinated tool calls and at most one fabricated citation per compliance run, and it scored at or near the top of the small-model quality battery. For a private assistant that chains tool calls against your own systems, or a grounded document assistant that must cite what it retrieved, that hygiene is worth real money in review time.

Pick the hardware to match the job. For a team of one to sixteen users on a single-box budget, one GB10 delivers 35.1 tok/s single-user and holds 16 concurrent users at the floor, at good energy efficiency per dollar. For a department service with dozens of simultaneous users, the GB300 measured 4,924.2 tok/s aggregate at 32 users, which is the configuration where this model stops feeling slow. On Strix Halo it fits in a laptop and runs at about 41 tok/s, which makes it the biggest model that comfortably fits that class, but serve it through ROCm on Windows, not Vulkan.

Look elsewhere when any of these apply: your prompts routinely exceed the low 100k range (its verified ceiling is the 131k-token model limit, and its 128k prefill could not be measured because the served context stops at the model limit), you need more than 16 concurrent users on a single GB10 at a responsive per-user rate, or you need the cheapest tokens per joule, where smaller MoE models measured several times better than its 1.63 tok/J.

Where to go next

• LLM benchmarks hub
• Leaderboards
• How we measure
• Model tier list
• Private AI solutions
• Private LLM guide
• CMMC compliance

FAQ