LLM Benchmark Results:Every Configuration Measured on Our Own GPUs
These LLM benchmarks are organized the way we actually use the results: one card per model and hardware configuration, with the engine, the quantization, the context, the single-user and concurrent throughput, the measured date and the trade-offs in the open. Where a community recipe or engine produced a result, the card says so. Where we did not measure something, the card says that too.
What These LLM Benchmarks Measure and Why It Matters
These LLM benchmarks measure how well a model does the work a regulated firm needs done, on hardware that firm can own, rather than how it scores on a public test set it may have been trained on. Public leaderboards rank models on academic corpora and often republish vendor numbers. A buyer who needs a model to fix a production script, answer a compliance question with a citation, or refuse an unsafe tool call gets little from a single aggregate score.
So Petronella Technology Group, Inc. built its own harness and ran it on its own fleet: eight NVIDIA GB10 appliances (four DGX Spark, four MSI EdgeXpert) on a switched 200G RoCE fabric, one DGX Station GB300, RTX PRO 6000 Blackwell workstation cards and a four-way H200 NVLink pool. Each configuration runs a fixed battery: tool-calling suites, agentic multi-turn tasks, a closed-book and a retrieval-grounded compliance set, coding, long-form writing and long-context needle tests, all cross-judged so no model is ever graded by a model from its own family. The rules behind that battery, and the conclusions we have retracted after auditing our own work, are on the benchmark methodology page.
Three rules make a card on this page trustworthy. First, every speed number is gated: an engine must pass arithmetic and needle-in-a-haystack checks before any throughput is recorded. Second, every number carries its serving configuration, because the wrong tool parser or a missing speculative-decoding flag changes a result more than a model swap does. Third, every card names the date it was measured and the stored result file it came from, so the age of every row is visible.
Why it matters: the models that win here are the models you can run inside your own compliance boundary. Coding and cited research on CUI, PHI or privileged material cannot be sent to a third-party API, so the practical question is not which model is best in the abstract but which configuration is best on the hardware tier you can justify. That is what the cards, the tier list and the cost table below answer.
Need one of these configurations running on hardware you own? Schedule a 15-minute private-AI call with Petronella Technology Group, Inc.
GLM-5.3-Flash on the DGX Station GB300: a 320B model, a 1M-token context, one box
One workstation-class box, no fabric, serving the same model our four-way H200 pool runs, with the largest verified context we have measured anywhere in the fleet.
Why it leads its tier. It is the only candidate on this box that is top or tied-top in every category that reflects real work: closed-book compliance 26 of 35 (25.6 mean over five runs), tools 38 of 38, agentic 15 of 15, zero hallucinated tool calls on the 27-case multi-turn corpus, HumanEval 97.0 percent, and all 26 long-context tasks passed. The nvidia NVFP4 checkpoint served a 968,377-token needle at 10, 50 and 90 percent depth (33 to 37 s end to end), about twice the context our H200 production service is configured for. One user on this box matches one user on four H200s.
The honest trade-offs. The 252 GB HBM cannot hold the FP8 checkpoint (328 GB), so this box runs a 4-bit quant, which measurably loses exact compliance recall: over five runs each, NVFP4 answered 121 of 175 compliance items against 138 for FP8, concentrated on exact recall such as requirement text to control ID. Feeding the NIST text with retrieval closes the gap (both variants score 26 of 26 on document-grounded tasks). Aggregate throughput plateaus near 1.7 to 2.2k tok/s from 32 users up, well below the H200 quad's 5.2k. And NVIDIA has published no MSRP for this machine; the about $85,000 figure in our cost table is the reported launch price, not a list price.
Every Configuration We measured, One Card Each
Each card is one model on one hardware configuration with the engine and quantization that produced the number: single-user and concurrent decode, prefill, context, the measured date and the trade-offs. Cards marked with a recipe credit ran a community-published engine or launch recipe; the credit section below names each one. Full ladders and the quality battery are on the leaderboards page.
Cards without a measured value in a field say so rather than estimate. Configurations we measured but do not card (three-unit and six-unit layouts, the DeepSeek-V4.1-Flash TensorFold test, the GLM-5.3 full offload run) are on the leaderboards page with their sources.
Platform Tier List: Which Configuration Is Worth Building
These tiers rank hardware configurations, not models, by measured value for a regulated private deployment: throughput per $1,000 at matched operating points, tokens per joule, the quality ceiling the configuration reached, and the data class or context depth it unlocks. A configuration can appear twice when two layouts of the same hardware behave differently. Every figure repeats a measured number from the cards and the leaderboards.
How to read the tiers (S is the best value we measured, C the weakest):
One GB10 appliance · the individual and small-team tier
Why S: the best affordability on the page. Nemotron-3.5-Lightning NVFP4 delivers 19.2 tok/s per $1,000 for a single user and 169 at its 64-user ceiling on a $4,699 list-price appliance, at 6.3 tok/J, and one unit served a full million tokens of context. For prefill-heavy work (retrieval, summarization, classification) one GB10 is a 4,600 to 6,900 tok/s machine.
The catch: pick the model for the work. Nemotron's quality is the weakest measured (29 of 38 tools, 3 of 35 closed-book compliance); Gemma-4-26B-A4B holds real quality (37 tools, 13 to 14 compliance, 24 of 26 long context) at 31 tok/s for one user, and grounded retrieval lifts it to 55 to 62 of 93 strict.
Four GB10 units on the 200G RoCE fabric · the production pod tier
Why S: the best combination of throughput, efficiency and quality on one budget. Qwen3.8-Flash-Next serves 617 to 649 tok/s across 64 users at 1.18 tok/J (34.5 tok/s per $1,000 at the ceiling); GLM-5.3-Flash on the same four units holds the best closed-book compliance of any GB10 layout (26 of 35) with zero hallucinated tool calls and a million-token context with a 3.9M KV pool.
The catch: the fabric is not optional (direct cables made a four-way cluster slower than one unit), and the two models trade speed for quality, so the pod runs whichever the workload needs.
Two GB10 units · the long-context pair
Why A: a pair buys depth, not speed. DeepSeek-V4-Flash-Vision-Exp on two units is the best two-unit single-user model measured (47.8 tok/s prose, tools 38, long context 26 of 26) and serves a million tokens. Qwen3.8-Flash-Next on a pair is the serving option: 329.6 tok/s across 32 users at 1.16 tok/J.
The catch: the Vision-Exp kit caps at 6 sequences (117 tok/s ceiling) and is the least efficient measured layout at 0.41 tok/J; one 26B-A4B unit already serves 64 users at the floor, so a pair is rarely worth it for throughput alone.
DGX Station GB300 · the single-box frontier tier
Why A: one user at 145 tok/s matches four H200s, with the largest verified context in the fleet (a 968,377-token needle found) and 32.9k tok/s prefill, all in one box with no fabric. The strongest closed-book compliance of the one-box candidates (26 of 35, the only correct SSP narrative among them).
The catch: it must run 4-bit (the FP8 checkpoint does not fit 252 GB), which measurably loses exact compliance recall (121 vs 138 of 175 over five runs); aggregate plateaus at 1.7 to 2.2k tok/s from 32 users up; and NVIDIA has published no MSRP, so its affordability row rests on the reported launch price, not a list price.
Four H200 NVLink · the concurrency tier
Why B, not S: nothing we measured serves more simultaneous users (5,245 tok/s at 256 users, still scaling at 128), runs FP8 natively with the best measured compliance mean (27.6 of 35), or carries frontier-size open models whole. But it is the most expensive tier (about $121,000 for the cards at a July 2026 market average, not an MSRP, plus the host server), it has no FP4 path, and its per-watt figure is not measured because the pool is not on metered outlets. For a single user it buys nothing over the GB300.
Eight GB10 units as two four-unit groups · the scaled pod
Why B: the same eight units, split into two independently measured four-unit groups, add up to about 1,266 to 1,298 tok/s across 128 users against 770 for one eight-way engine. Those totals are arithmetic from the measured group rows at matched load, not one concurrent test (the groups sit on separate switches, and earlier concurrent runs showed no cross-group interference, but we have not re-run both groups at once since the October 3 circuit trip).
Eight GB10 units as one eight-way engine · only for single-user speed
Why C: the worst value per dollar on the page for anything but one user at a time: 20.5 tok/s per $1,000 at 64 users (770 tok/s) against 34.5 for each four-unit group. An eight-way group pays communication overhead across the whole fabric on every token while two four-way groups stay local and take separate queues. Its one win is single-user speed, 45.7 tok/s. Build this layout only when one user must have the whole pod.
Results by Model: One Page per Model, Same Measured Fields
Each model page carries the same measured fields: what the model is, where it runs best, its measured speed on our fleet, quality and accuracy, how we ran it, when to pick it, and an FAQ. Numbers on a model page trace to the same stored result files as the cards and tables above.
DeepSeek-V4.1-Flash
Gemma 4 26B
GLM-5.3-Flash
gpt-oss-120b
MiMo-V2.6-Flash-RL
Nemotron-3.5-Lightning
Qwen3.6-35B-A3B
Qwen3.8-27B
Qwen3.8-Flash-Next
Download the data: llm-benchmarks.csv or llm-benchmarks.json - every record and metric with its source path.
Affordability: Throughput per Dollar and per Joule, With the Price Basis Stated
Affordability is part of the honest picture, so it is measured, not implied. Prices are list prices where a manufacturer publishes one and clearly labeled market figures where it does not. GPU prices vary with market conditions; treat every per-dollar row as a snapshot of the stated basis, not a quote.
| Platform | Model | Price basis | 1 user, tok/s per $1k | Ceiling, tok/s per $1k |
|---|---|---|---|---|
| 1x GB10 | Nemotron-3.5-Lightning NVFP4 | $4,699 NVIDIA list (DGX Spark, hardware only) | 19.2 (90.2 tok/s) | 169 at 64 users (795.4 tok/s) |
| 2x GB10 | Qwen3.8-Flash-Next NVFP4 | $9,398, 2x list | 3.7 (35.2 tok/s) | 35.1 at 32 users (329.6 tok/s) |
| 4x GB10 | Qwen3.8-Flash-Next NVFP4 | $18,796, 4x list, fabric excluded | 2.3 (42.5 tok/s) | 34.5 at 64 users (648.7 tok/s) |
| 8x GB10, one engine | Qwen3.8-Flash-Next NVFP4 | $37,592, 8x list, fabric excluded | 1.2 (45.7 tok/s) | 20.5 at 64 users (769.7 tok/s) |
| 8x GB10, two 4-unit groups | Qwen3.8-Flash-Next NVFP4 | $37,592, 8x list | same as 4-unit rows | about 33.7 at 128 users total (projected: the sum of two separately measured groups) |
| DGX Station GB300 | GLM-5.3-Flash NVFP4 | No published MSRP; about $85,000 reported launch price, which is not an MSRP | 1.7 (145 tok/s, no MTP) | 16.2 at 32 users (1,380 tok/s) |
| 4x H200 NVL | GLM-5.3-Flash FP8, MTP3 | No published MSRP; about $121,000 for four cards, July 2026 market average, cards only | 2.0 (238 tok/s) | 18.6 at 32 users (2,254 tok/s) |
What the price rows hide, stated plainly. Four of the eight clustered GB10 units are MSI EdgeXpert systems, priced here at the DGX Spark list price because MSI has published no price of its own. Fabric is excluded from the cluster rows: a pair needs one cable, four and eight units need a switch and breakout cables, so the cluster rows are, if anything, flattered by the exclusion. The DGX Station has no published MSRP at all, so its row uses the reported launch price and is labeled as such. The H200 row excludes the host server the cards live in. NVIDIA's DGX Spark list price has held at $4,699 since February 2026; card and appliance prices move with market conditions, so re-check the basis before using any row in a purchase decision.
| Platform | Model | Wall W, measured | tok/J at the stated point |
|---|---|---|---|
| 1x GB10 | Nemotron-3.5-Lightning | 111 W (1 user) / 126 W (64-user ceiling) | 6.30 at the ceiling |
| 4x GB10 | Qwen3.8-Flash-Next | 439 W (1 user) / 535 W (32 users) | 0.90 at 32 users; 1.18 at 64 |
| 4x GB10 | GLM-5.3-Flash | 507 W (1 user) / 579 W (32 users) | 0.45 at 32 users |
| 8x GB10 | MiMo-V2.6-Pro-RL | 1,169 W (1 user) / 1,179 W (8 users) | 0.083 at 8 users (prose) |
| DGX Station GB300 | GLM-5.3-Flash | not measured on a metered outlet in these runs | not measured |
| 4x H200 NVL | GLM-5.3-Flash FP8 | not measured on a metered outlet in these runs | not measured |
The efficiency spread is large and it is mostly architecture: the one-unit MoE models turn wall power into tokens about 75 times better than the eight-unit MiMo layout (6.30 against 0.083 tok/J), because bandwidth-bound appliances with few active parameters are the efficient point of this fleet and a 535 GB eight-way model is not. On the MSI units, letting the clocks boost to default raised wall power 10 to 23 percent and cut tokens per joule 12 to 18 percent on the multi-user tests while changing throughput by about 1 percent, which is why the locked-clock rows are the ones we serve.
Which tier you can justify depends on the memory pool your target model needs, the number of concurrent users at peak and the data class you handle. Our on-premise AI server builds page covers the platforms behind each tier, and the self-hosted LLM planning guide covers the decisions that come before hardware.
Decode and Prefill, the Numbers Behind the Cards
Single-row summaries here; the full per-user and per-prompt ladders, including the configurations without a card, are on the leaderboards page.
| Platform and model | 1 user | 32 users | Measured peak |
|---|---|---|---|
| 1x GB10, Nemotron-3.5-Lightning | 90.2 | 608.0 | 795.4 at 64 users |
| 1x GB10, Gemma-4-26B-A4B | 31.5 | 552.8 | 866.4 at 64 users |
| 2x GB10, Qwen3.8-Flash-Next | 35.2 | 329.6 | 329.6 (32-user cap of the ladder) |
| 2x GB10, DeepSeek-V4-Flash-Vision-Exp | 47.8 | not measured (6-request profile) | 117.0 at 6 users |
| 4x GB10, GLM-5.3-Flash | 44.9 | 258.6 | 258.6 (32-user cap of the ladder) |
| 4x GB10, Qwen3.8-Flash-Next | 42.5 | 486.8 | 617 to 649 at 64 users |
| 8x GB10, MiMo-V2.6-Pro-RL (published workload) | 99.25 (1 stream) | not measured at 32 (8-slot profile) | 312.1 at 8 streams |
| 8x GB10, Qwen3.8-Flash-Next (one engine) | 45.7 | 604.7 | 769.7 at 64 users |
| DGX Station GB300, GLM-5.3-Flash (no MTP) | 140 to 145 | 1,380 | 2,517 at 256 users |
| 4x H200 NVL, GLM-5.3-Flash FP8 (MTP3) | 227 to 238 | 2,254 | 5,245 at 256 users |
| 4x RTX PRO 6000, Qwen3.8-Flash-Next | 190 to 229 | 1,586 | 1,586 (32-user cap of the ladder) |
| Platform and model | 8k | 32k | 128k |
|---|---|---|---|
| 1x GB10, Nemotron-3.5-Lightning | 6,859 | 6,431 | 4,632 |
| 4x GB10, Qwen3.8-Flash-Next | 3,571 | 3,530 | n/a (131k served context) |
| 4x GB10, GLM-5.3-Flash | 2,120 | 2,049 | 2,048 |
| 8x GB10, MiMo-V2.6-Pro-RL | 2,009 | 1,722 | 1,600 (1,378 at 257k) |
| DGX Station GB300, GLM-5.3-Flash | 32.9k | 32.6k | 32.0k |
| 4x H200 NVL, GLM-5.3-Flash FP8 | 20.8k | 22.7k | 22.0k at 131k (20.7k at 259k) |
The two ladders tell opposite stories, and both matter when sizing a deployment. Prefill is compute-bound and it is where the GB300 is untouchable: 32.9k tok/s at an 8k prompt is about nine times the four-unit GB10 pod. Decode is bandwidth-bound and it is where the appliance pod earns its keep per dollar. A retrieval workload (long prompt, short answer) and a drafting workload (short prompt, long answer) can land on different platforms, which is exactly why both ladders are published.
Checked, Not Assumed: What Stands Behind Every Number
A number earns a card only after the checks below. Where a check failed or stayed inconclusive, we publish that too, because a failed check is information a buyer needs.
A correctness gate runs first, and a failed gate blocks the row.
Every engine starts from a fresh reboot of its units with a 20-second burn verify (no 611 MHz clock latch was seen on any unit in any run). Before any throughput is recorded, the engine must pass arithmetic and needle-in-a-haystack checks to 120k tokens; the GB10 tier runs passed the gate 8 of 8 in both clock states. A wall-power meter (rack PDU, per unit) and one-second GPU telemetry are recorded during each run.
Advertised context is not served context, so every context row is needle-verified.
The DGX Station found a needle in a 968,377-token prompt at 10, 50 and 90 percent depth (33 to 37 s each). Nemotron-3.5-Lightning served a full 1,048,576 tokens but its 983,609-token needle produced no verbatim answer within the 4,096-token budget, so we call that row "served, not needle-verified". The two-unit GLM engine accepted 655,360 tokens (a 786,432 setting was OOM-killed by the kernel), but a roughly 589k-token prompt killed the engine on an executor timeout, so the pair is honestly a 262k, 6-request GLM until the timeout knob is re-tested. We publish the failure because it is the difference between what an engine accepts and what it completes.
Same-clock parity runs catch hardware drift before it contaminates a comparison.
At an equal 2,300 MHz clock the DGX Spark and MSI EdgeXpert builds decoded within 2 percent on every one-unit and two-unit point (the MSI drew 4 to 8 W more per unit and ran 1 to 8 C warmer). Locked-clock versus default-boost changed multi-user throughput by about 1 percent while raising wall power 10 to 23 percent. Zero power-loss events and zero clock latches were recorded across both states.
No model is ever graded by a model from its own family.
Self-judging inflates scores 20 to 37 percent in our measurements, so every quality row is cross-judged: the generator never shares a family with the judge, headline studies use a three-model frontier panel with the median taken per case, and longitudinal rows use a locked campaign judge with the temperature and rep count stamped on each. The methodology page documents the judge lock, the metadata every result file must carry, and the four conclusions we retracted after auditing our own staged claims.
Standing on Community Recipes, Named Where Used
Several results on this page run a community-published engine, checkpoint or launch recipe. The credit belongs on the card, so here is the complete list of which result used whose work. Everything else on the page ran NVIDIA's own published recipes or our own launch configurations.
- Mia (Mia's AI Lab): the two-unit DeepSeek-V4-Flash-Vision-Exp card ran her Anemll DGX Spark kit, and the DeepSeek-V4-Flash-0731 forum lanes ran the AlexLJC/MiaAI kit. Her published work at github.com/MiaAI-Lab.
- TensorFold, by Ash Hart: the DeepSeek-V4.1-Flash-1017 two-unit and four-unit tests (October 18 to 19) ran Tony's TensorFold builds of the LibertAI EXL3 checkpoints. The engine is at github.com/ashhart/TensorFold.
- BertholomusAI: the September 25 lane comparing DeepSeek-V4-Flash-0731 at TP4 against two TP2 pairs ran BertholomusAI's recipe.
- tonyd2wild: the GLM-5.3-Flash GB10 results run his v11 vLLM build with the DFlash2 block drafter.
- oxbyte: the Qwen3.8-Flash-Next pod results use his MTP4 and expert-parallel serving configuration.
- rhys101: the MiMo-V2.6-Pro-RL card runs his SGLang a14 recipe with the DFlash-8 adaptive drafter.
- NVIDIA: the one-unit Nemotron, Qwen3.6 and Gemma cards run NVIDIA's published DGX Spark recipes and NVFP4 checkpoints.
Where our numbers differ from a recipe's published numbers, both are stated (the MiMo card reproduces its published speed within 5 percent; the TensorFold DeepSeek run measured about 8 percent under its published single-stream figure). A recipe credit means the launch configuration came from the community author; every measurement, quality score and verdict on this page is ours.
LLM Benchmarks FAQ
Short answers to the questions buyers ask most; every figure repeats a result shown above.
What do these benchmarks actually measure?
Can a model grade its own output?
What hardware were these results measured on?
Does adding units make a single user faster?
Why do some cells say "not measured"?
How are prices handled?
Which model should run for compliance work?
Is 4-bit quantization safe for compliance answers?
Why does a card name a recipe author?
Can Petronella Technology Group, Inc. run one of these configurations inside my compliance boundary?
About the Author
Craig Petronella is a CMMC Registered Practitioner (CMMC-RP), Cisco CCNA, CWNE and holder of Digital Forensic Examiner License 604180-DFE. He is the Amazon #1 best-selling author of 14+ cybersecurity books and the founding principal of Petronella Technology Group, Inc., which has served regulated and defense-adjacent organizations since 2002.
Petronella Technology Group, Inc. is a Cyber AB Registered Provider Organization (RPO-1449). Every number on this page was produced on hardware the company owns and operates in Raleigh, North Carolina, using the harness described in the methodology section; no vendor-supplied numbers are used, and the losing configurations and failed checks are published beside the winners.
Put the Results to Work
The benchmark tells you which configuration. These guides cover the rest: readiness, policy, compliance scope and how a private deployment is run day to day.
The Business AI Guide
Where private AI pays off, what it costs, and how to start without exposing data. Free download.
AI Readiness Checklist
The questions to answer before you buy hardware or sign an API agreement.
CMMC Compliance Hub
Why CUI workloads drive on-premises AI, and what an assessor expects to see.
AI Cybersecurity Solutions
Securing the models, the data pipeline and the hardware they run on.
Books by Craig Petronella
Practical references on CMMC, HIPAA and cybersecurity for business owners.
On-Premise AI Server Builds
The platforms behind every tier on this page, from one appliance to a datacenter pool.
Talk to the team that measured every number on this page
A 15-minute call is enough to tell you which tier fits your users, your data and your budget, and what it will cost to run.
Petronella Technology Group, Inc. · 5540 Centerview Dr., Suite 200, Raleigh, NC 27606 · 919-348-4912 · info@petronellatech.com · Last Updated: October 23, 2026