GB10 (DGX Spark) vs H200 vs RTX PRO 6000 for LLM Inference:Measured Side by Side
This page compares three hardware families where our measurements overlap: the GB10 (DGX Spark class) appliances alone and in pods, the RTX PRO 6000 Blackwell Max-Q workstation server (one, two and four GPU configurations), and the four-way H200 NVL server. We report single-user decode speed, aggregate throughput under concurrent load, prefill speed and verified context for the same models across tiers, then say who should buy which. Where a model was measured on only one tier, we say so instead of projecting.
What We Compared and How We Measured It
The GB10, the RTX PRO 6000 and the H200 sit at three points on the private-inference ladder we run for clients. The GB10 is a desk-side appliance: one small box with unified memory that an individual team can own outright, run on a normal office circuit and move between offices. The RTX PRO 6000 Blackwell Max-Q is a workstation-class GPU; our server holds four of them, and we have measured one-GPU, two-GPU and four-GPU configurations on that box, with each GPU carrying 96 GB of memory. The H200 NVL is a data-center accelerator; our pool is four of them on NVLink in a separate production server, and it serves production traffic for our own agents every day.
For this comparison we only used configurations where the model was measured on the tier in question with our standard battery: a fixed prompt set for single-user prose and code decoding, a concurrency ladder that walks user counts up to the engine's ceiling, prefill timing at three prompt lengths, long-context needle tests, tool-calling suites and our retrieval-grounded compliance set. Every speed number is gated: the engine must pass arithmetic and needle checks before any throughput is recorded. The full rules, the gate details and the conclusions we have retracted after auditing ourselves are on the methodology page, and the complete ladders behind every table here are on the leaderboards page.
Two honest caveats before the numbers. First, the quantizations differ by family: the GB10 results below use NVFP4 or MXFP4 checkpoints because that is what fits and runs well on that class of hardware, while the RTX PRO 6000 and H200 results mostly use FP8 or MXFP4. Quantization is part of the configuration, so the comparison is "the best configuration we have measured on each tier for this model," not a like-for-like precision test. Second, concurrency ceilings differ: the GB10 appliances saturate somewhere between six and sixty-four users depending on the model, while the RTX PRO 6000 and H200 configurations were laddered to much higher user counts. We report each tier at its own measured ceiling and at matched user counts where both exist.
Want this comparison run against your own workloads? Schedule a test on our fleet
Single-User Decode Speed, Same Model Across Tiers
Single-user decode is the number a developer or an analyst feels directly: how fast a private model writes back. Prose tokens per second, one stream, measured date for each row.
| Model | GB10 configuration | tok/s | RTX PRO 6000 configuration | tok/s | H200 NVL configuration | tok/s |
|---|---|---|---|---|---|---|
| GLM-5.3-Flash | 4x GB10 pod, vLLM, NVFP4, DFlash2 | 44.9 | 2x RTX PRO 6000, vLLM FP8 | 166.0 | 4x H200 NVL, vLLM FP8, MTP3 | 238 |
| Qwen3.8-27B | 1x GB10, vLLM, NVFP4 | 24.4 | 1x RTX PRO 6000, vLLM FP8 | 80.1 | not measured | not measured |
| gpt-oss-120b | 1x GB10, vLLM, MXFP4 | 35.1 | 1x RTX PRO 6000, vLLM MXFP4 | 179.0 | not measured | not measured |
| DeepSeek-V4.1-Flash | 4x GB10 pod, vLLM, NVFP4 | 54.1 | 4x RTX PRO 6000, vLLM NVFP4, TP4 + EP | 61 | not measured | not measured |
Source: fleet-bench summaries for the GB10 runs of 2026-10-02 and 2026-10-03, the RTX PRO 6000 records measured on h200monsta between 2026-09-21 and 2026-10-10, and the H200 NVL records measured on the production pool; the stored records are in data/llm-bench/results/gb10.json, rtx-pro-6000.json and h200.json. GLM-5.3-Flash is the only model in this table with records on all three tiers, so the other rows say "not measured" where we have no record instead of projecting one.
Three patterns matter. First, the gap between GB10 and RTX PRO 6000 is real but model-dependent: Qwen3.8-27B runs about three times faster per stream on one RTX PRO 6000 GPU than on one GB10, and gpt-oss-120b about five times faster, while DeepSeek-V4.1-Flash nearly closes the gap on the four-GPU configurations, because that model is a sparse mixture-of-experts whose per-token work is small relative to the memory traffic both tiers can supply. If the model you want is DeepSeek-class, a GB10 pod is a much less painful downgrade than the headline gap suggests.
Second, speculative decoding changes the ranking on the GB10 side. GLM-5.3-Flash on the four-unit pod reaches 44.9 tok/s per stream with the community DFlash2 recipe, which is respectable for a model of that size. On two RTX PRO 6000 GPUs the same model reaches 166.0 tok/s with plain FP8 serving, and on the four-GPU H200 NVL pool it reaches 238 tok/s with the MTP3 speculative recipe on top of FP8. On the bigger tiers the hardware does the work; on the GB10 the recipe does part of it. Both are legitimate, but they age differently: a recipe tied to one checkpoint version can break on the next update, while raw FP8 serving survives it.
Third, note what the two-GPU numbers say about scaling. GLM-5.3-Flash on two RTX PRO 6000 GPUs runs 166.0 tok/s per stream, and on a two-GB10 pod runs 24.1 tok/s, yet a single GB10 running smaller models beats both of those configurations when the model fits: Nemotron-3.5-Lightning-30B-A3B on one GB10 runs 90.2 tok/s for prose and 114.2 tok/s on code prompts. Matching the model to the hardware tier matters more than buying more of the smaller unit.
For reading speed rather than generation speed, these numbers translate directly: at 80 tok/s a page of prose arrives in a couple of seconds; at 24 tok/s you are watching text type out. For chat assistants and coding copilots, where the user reads while the model writes, everything from 30 tok/s upward on the GB10 feels usable, and everything on the RTX PRO 6000 and H200 tiers feels instant.
Aggregate Throughput: Concurrency Is Where the Bigger Tiers Pull Away
Aggregate decode throughput is the sum across all concurrent streams: the number that decides how many people a deployment serves. Where tiers were laddered at the same user count, we show that row.
| Model | GB10 aggregate | RTX PRO 6000 aggregate | H200 NVL aggregate |
|---|---|---|---|
| gpt-oss-120b, 32 users | 290.5 tok/s on 1x GB10 (ceiling 16 users, 198.0 tok/s) | 1,118.1 tok/s on 1x RTX PRO 6000 (ceiling 64 users, peak 1,511.2) | not measured |
| Qwen3.8-27B, 32 users | 316.6 tok/s on 1x GB10 | 943.2 tok/s at 16 users on 1x RTX PRO 6000; 1,190.2 tok/s at 48 users; peak 1,358.0 | not measured |
| GLM-5.3-Flash, 32 users | 258.6 tok/s on the 4x GB10 pod; 174.3 tok/s at its 16-user ceiling | 394.0 to 454.0 tok/s at the eight-stream and sixteen-stream points on 2x RTX PRO 6000 | 2,254 tok/s on 4x H200 NVL; peak 5,245 tok/s |
| DeepSeek-V4.1-Flash, 32 users | 125.6 tok/s at 4 streams on the 2x GB10 TensorFold recipe | 583 tok/s on 4x RTX PRO 6000 | not measured |
Source: the same stored records as the single-user table, plus the concurrency ladders in each family file. The single-GPU Qwen3.8-27B ladder on the RTX PRO 6000 server also measured 568.8 tok/s at eight users; the H200 NVL GLM-5.3-Flash run verified 510,997 tokens of context while serving.
Aggregate throughput is the number that sizes a deployment, and it is where the tiers stop being comparable in kind. One GB10 appliance serving gpt-oss-120b tops out at 198.0 tok/s across its sixteen-user ceiling; a single RTX PRO 6000 GPU serving the same model sustains 1,118.1 tok/s at thirty-two users and peaked at 1,511.2 tok/s in our ladder. That is not a tuning gap we could close with a better engine: the workstation GPU simply has more memory bandwidth to feed concurrent streams.
The practical reading is a head-count rule. A GB10 appliance comfortably serves a small team: one user editing code, a handful of chat sessions, a nightly batch job. Past roughly its ceiling, per-user speed collapses because the queues fill; on gpt-oss-120b, aggregate throughput actually falls from 290.5 tok/s at thirty-two requested streams to 198.0 tok/s at the sixteen-user ceiling. A RTX PRO 6000 GPU absorbs a department on one card and still had headroom at sixty-four users. The four-way H200 NVL pool kept absorbing users beyond that: our GLM-5.3-Flash production service ran 2,254 tok/s aggregate at thirty-two users and reached 5,245 tok/s at its peak, which is why it runs our own voice and console agents all day.
There is one more number worth naming: serving headroom is not just speed, it is verified context under load. The four-way GLM-5.3-Flash configuration on the H200 NVL pool held a verified context of 510,997 tokens while serving concurrent traffic, and its KV cache capacity was measured at 3,329,280 tokens. On the RTX PRO 6000 server, the four-GPU DeepSeek-V4.1-Flash configuration verified 163,840 tokens with a KV cache capacity of 763,467 tokens. The GB10 appliances verified between roughly one hundred twenty thousand and two hundred forty-three thousand tokens of usable context per configuration, which is plenty for document work and not enough for whole-repository or whole-case-file sessions.
Prefill Speed and Verified Context
Prefill is how fast the model reads a prompt before the first token comes back: it decides how long a user waits when pasting a long document. Context is how much of your material fits in one session.
| Model and tier | Prefill, 8k prompt | Prefill, 32k prompt | Prefill, 128k prompt | Verified context |
|---|---|---|---|---|
| Nemotron-3.5-Lightning, 1x GB10 | 6,859 tok/s | 6,431 tok/s | 4,632 tok/s | not measured |
| Qwen3.6-35B-A3B, 1x GB10 | 7,492 tok/s | 6,021 tok/s | 3,117 tok/s | not measured |
| gpt-oss-120b, 1x GB10 | 4,900 tok/s | 4,037 tok/s | not measured | 120,755 tokens |
| Gemma-4-26B-A4B, 1x GB10 | 6,247 tok/s | 2,689 tok/s | 784 tok/s | 243,271 tokens |
| Qwen3.8-27B, 1x GB10 | not measured | not measured | not measured | 243,392 tokens |
| Qwen3.8-27B, 1x RTX PRO 6000 | not measured | not measured | not measured | 131,072 tokens |
| DeepSeek-V4.1-Flash, 4x RTX PRO 6000 | not measured | not measured | not measured | 163,840 tokens |
| GLM-5.3-Flash, 4x H200 NVL | 20,763.5 tok/s | 22,696.7 tok/s | 21,999.8 tok/s | 510,997 tokens |
Source: the prefill ladders and verified-context records in the same family files. Where we did not run a length or a tier, the cell says so; we do not estimate.
On prefill the H200 NVL pool is in a different class from everything we measured beside it. The four-way GLM-5.3-Flash configuration read an eight-thousand-token prompt at more than twenty thousand tokens per second and, unusually, did not slow down as prompts grew: it sustained roughly the same prefill rate at the 32k and 128k lengths, around twenty-two thousand tokens per second. That flatness matters for real work, because it means time-to-first-token stays predictable as sessions grow. The GB10 appliances prefill fastest on short prompts, up to about seven and a half thousand tokens per second on the best one-unit configurations, and then fall off as prompt length grows, sharply so on models with small active expert counts: Gemma-4-26B-A4B dropped from 6,247 tok/s at 8k to 784 tok/s at 128k.
On verified context the H200 NVL pool wins by a wide margin where it matters most. The GLM-5.3-Flash run found needles at a verified 510,997 tokens. The largest verified contexts elsewhere are on the GB10 side, where the Qwen3.8-27B run reached 243,392 tokens and the Gemma-4 run 243,271 tokens, with gpt-oss-120b verified at 120,755 tokens on one GB10. On the RTX PRO 6000 server, the Qwen3.8-27B one-GPU run verified 131,072 tokens and the four-GPU DeepSeek-V4.1-Flash run 163,840 tokens. For compliance work where an entire policy library, or an entire engagement folder, needs to be in one prompt, the H200 NVL tier is the only one of the three that has demonstrated half a million tokens.
The latency side of prefill shows up in the time-to-first-token measurements too. On one RTX PRO 6000 GPU, gpt-oss-120b returned the first token of a three-thousand-token judged prompt in a median of 0.904 seconds for a single user and 7.99 seconds with thirty-two users queued; the median prompt in that test was 3,098 tokens. On two RTX PRO 6000 GPUs serving GLM-5.3-Flash, the Sam-turn workload returned a first token in a median of 0.14 seconds in the fastest recipe we swept. On the GB10 appliances we did not record the same TTFT statistic, so we report what we have rather than filling the gap with a guess.
Quality on the Work That Matters: Tools, Agents and Compliance
Fast wrong answers are worse than slow right ones, so the tiers run the same quality battery where the models overlap. On our retrieval-grounded compliance suite, Qwen3.8-27B scored 0.971 strict on one RTX PRO 6000 GPU with a 0.925 code-first mean, while its GB10 NVFP4 run recorded an agentic strict score of 0.735 under the stricter agentic protocol. gpt-oss-120b scored 0.76 strict on one RTX PRO 6000 GPU in MXFP4 with a 0.889 code-first mean, against 77 percent strict on one GB10, essentially the same. On the four-GPU H200 NVL pool, GLM-5.3-Flash scored 0.964 strict with a 0.928 code-first mean, and on a two-GB10 pod with the DFlash2 recipe it scored 0.978 on the agentic strict protocol, which makes GLM-5.3-Flash the standout quality-per-watt choice on the GB10 tier.
Tool calling tells a similar story. On the four H200 NVL GPUs, GLM-5.3-Flash passed 23 of 23 tool-call cases with 85.2 percent overall accuracy, a 0.917 agentic completion rate and zero hallucinated tool calls, and passed 38 of 38 on the deeper tool suite. On the RTX PRO 6000 server, the four-GPU DeepSeek-V4.1-Flash configuration also passed 38 of 38 on the deeper tool suite. On the GB10 side, the DeepSeek-V4.1-Flash TensorFold recipes passed 20 of 20 tool-call cases on both engine variants. The pattern we keep seeing: at current checkpoint quality, the small-model tier loses to the big tiers on ceiling, not on competence. A GB10 pod does the compliance-grounded and tool-driven work we ask of it; it just cannot do it for forty people at once.
One caution from the same data: quantization is a quality variable, not a constant. The DeepSeek-V4.1-Flash EXL3 recipe on a two-GB10 pod reached 91 of 93 correct on the agentic compliance set for one seed, and the native TensorFold variant of the same checkpoint reached 87 of 93. Ten points of score moved with a serving-recipe change, no hardware change at all, which is why every card on our pages names the engine and the quant next to the number.
Power, Placement and Cost Shape the Decision as Much as Speed
The GB10 appliances are efficient in a way the bigger tiers are not. Nemotron-3.5-Lightning on one GB10 measured 6.3 tokens per joule, and Qwen3.6-35B-A3B measured 3.62 tokens per joule on the same class of hardware; the eight-unit MiMo-V2.6-Pro-RL pod drew a measured 1,169 watts across all eight units while decoding. The GB10 runs on a normal office circuit, needs no rack, and can sit next to the compliance data it works on. The RTX PRO 6000 server sits between those worlds: a workstation-class box whose clean single-GPU baseline recorded copy bandwidth of 1,468 GB/s and read bandwidth of 1,633.1 GB/s at 300 watts for one card, before the other three cards and the cooling that keeps them fed. The H200 NVL pool is a data-center asset that adds the NVLink fabric and the facility power behind it.
That difference in placement is often the deciding factor, not the benchmark table. A two-person development team that handles sensitive client material can put a GB10 under a desk this week and keep every token inside their own boundary. A firm that has outgrown the desk but not the office can step up to the RTX PRO 6000 tier on one card. A firm that needs to serve an entire department, or that wants to run an always-on agent fleet like ours, needs the H200 NVL tier's memory bandwidth and its verified half-million-token context, and needs somewhere to put it.
Who Should Buy Which
- A single team up to roughly a dozen people runs one or two models, and the models are small or mixture-of-experts.
- The box must live in an office, on a normal circuit, with no rack and no facilities work.
- Deskside privacy is the point: code, documents and client material that must never leave the room.
- You want the best tokens per joule, and the workload tolerates per-user speeds from the mid twenties to about ninety tok/s.
- More than a dozen concurrent users or an always-on agent fleet share one model; one RTX PRO 6000 GPU sustained 1,118.1 tok/s aggregate at thirty-two users, and the four-GPU H200 NVL pool sustained 2,254 tok/s.
- Sessions routinely hold very long material: the H200 NVL pool demonstrated 510,997 verified tokens of context, not promised.
- Prefill latency on long documents must stay flat: the four-GPU H200 NVL pool held about twenty-two thousand tokens per second from 32k to 128k prompts.
- You have a rack, power and cooling, and throughput per dollar of facility matters more than watts per desk.
The tiers are complements, not rivals: most of our clients start with a GB10 pilot, step up to a RTX PRO 6000 card when one department outgrows the desk, then size an H200 NVL class pool once the workload's concurrency and context needs are measured on their own traffic.
If you are choosing for one person, the honest answer is that a single GB10 running the right model is enough: Nemotron-3.5-Lightning at 90.2 tok/s per stream is faster to read than most people read, and its 795.4 tok/s aggregate at the sixty-four-user ceiling means the same box absorbs a team spike. If you are choosing for a department, one RTX PRO 6000 GPU already carried gpt-oss-120b at 1,118.1 tok/s aggregate with 179.0 tok/s for a single stream. If you are choosing for an agent fleet, the H200 NVL pool is the only one of the three that held 2,254 tok/s of aggregate decode at thirty-two users with 510,997 tokens of verified context, and that difference is structural.
The model pages carry the per-model detail behind this comparison. Start with GLM-5.3-Flash, which is measured on all three tiers, and gpt-oss-120b, whose single-GPU numbers show what one workstation card does for one model. For the mixture-of-experts models that make the GB10 tier shine, see Nemotron-3.5-Lightning and DeepSeek-V4.1-Flash; for the dense small model, see Qwen3.8-27B. The leaderboards page holds the full concurrency ladders, and the hub ties every family together on the LLM benchmarks overview.
Test the model you actually want to run
Petronella Technology Group, Inc. runs these measurements on hardware we own, and we can run your candidate model and your workload shape on the tier you are considering before you spend anything. Call Penny at 919-348-4912, or use the button to book a benchmark slot.
GB10 vs H200 vs RTX PRO 6000: Frequently Asked Questions
Is a GB10 just a slow H200?
No. It is a different point on the curve. Per stream, one RTX PRO 6000 GPU is roughly three to seven times faster than a single GB10 for the same model, and the four-GPU H200 NVL pool is faster still for GLM-5.3-Flash, but the GB10 draws a fraction of the power, needs no rack and delivers the best tokens per joule we have measured in the fleet, including 6.3 tokens per joule for Nemotron-3.5-Lightning on one unit.
How many users can one GB10 serve?
It depends on the model. gpt-oss-120b on one GB10 saturated at sixteen users, where aggregate throughput measured 198.0 tok/s; Nemotron-3.5-Lightning held a sixty-four-user ceiling with 795.4 tok/s aggregate. Plan around the model's measured ceiling, not a generic user count.
Can a GB10 pod catch up to the H200 pool?
Not on aggregate throughput. Our four-unit GB10 pod running GLM-5.3-Flash measured 258.6 tok/s at thirty-two users against 2,254 tok/s for the four-GPU H200 NVL pool on the same model, and the H200 NVL pool peaked at 5,245 tok/s. Pods help availability and scaling per model, but the memory bandwidth gap is structural.
Which models were measured on which tiers?
GLM-5.3-Flash has records on all three tiers, which is what makes its rows the fairest on this page. Qwen3.8-27B, gpt-oss-120b and DeepSeek-V4.1-Flash have records on the GB10 and RTX PRO 6000 tiers; DeepSeek-V4.1-Flash also has a four-GPU RTX PRO 6000 record. Models such as Nemotron-3.5-Lightning and Qwen3.8-Flash-Next were measured on the GB10 tier only, and we mark them as not measured on the other tiers rather than projecting a number.
Is the quality lower on the GB10 because of quantization?
For the models we tested, mostly no, but it depends on the recipe. gpt-oss-120b scored 77 percent strict on one GB10 in MXFP4 against 0.76 on one RTX PRO 6000 GPU in the same quant, and GLM-5.3-Flash on a two-GB10 pod scored 0.978 agentic strict. The one gap we measured was exact compliance recall between precision variants, which retrieval closes.
What about long documents and big context windows?
The H200 NVL tier is the clear winner where verified: 510,997 tokens of context on the four-way GLM-5.3-Flash service. On the RTX PRO 6000 server, 163,840 tokens are verified for the four-GPU DeepSeek-V4.1-Flash run and 131,072 for one-GPU Qwen3.8-27B. The best GB10 verified contexts are 243,392 tokens for Qwen3.8-27B and 243,271 for Gemma-4-26B-A4B, both comfortably above one hundred thousand but not half a million.
Does prefill speed differ as much as decode speed?
Yes, and it matters for pasted documents. The four-GPU H200 NVL pool prefilled an eight-thousand-token prompt at 20,763.5 tok/s and held about twenty-two thousand tokens per second at the 32k and 128k lengths, while the GB10 appliances range from roughly four thousand nine hundred to seven and a half thousand tokens per second at 8k and slow down as prompts grow. We have no prefill-ladder record on the RTX PRO 6000 tier, so we do not quote one.
Can I mix the tiers?
That is what we do. Sensitive, latency-tolerant single-user work runs on GB10 hardware near the data, the RTX PRO 6000 server carries department-scale serving, and the shared services that need the highest concurrency and the longest context run on the H200 NVL pool. The right split for your firm falls out of measuring your own concurrency and prompt lengths, which is exactly what our benchmark service does.
How current are these numbers?
Every number carries its measured date in the stored records, and the GB10 results here are from the 2026-10-02 and 2026-10-03 runs with a 2026-10-18 and 2026-10-23 recipe update for DeepSeek-V4.1-Flash, the RTX PRO 6000 results are from runs between 2026-09-21 and 2026-10-10, and the H200 NVL results are from the 2026-09-21 through 2026-10-10 runs. We republish this page when a newer measurement supersedes a row, and the leaderboards name the source file for every row.
Where can I see the full per-model data?
The benchmark hub links every model page and the leaderboards, and the downloadable data file on the hub carries every record with its source path. The methodology page explains the gates a result must pass before it appears anywhere.
Decide from measurements, not vendor slides
Bring us the model and the workload you need to keep private, and we will measure it on the GB10 tier and the bigger server tiers side by side, then hand you the numbers this page is made of.
Measured on the Petronella Technology Group, Inc. fleet. Every number on this page traces to a stored result file named in the source notes.