Benchmark · DGX Station GB300 · DGX Spark / GB10 · H200 · RTX PRO 6000 · Ryzen AI Max+ (Strix Halo)

GLM-5.3-Flashbenchmark results

This page holds every number we measured for this model: single-user prose speed of 238 tok/s on 4x H200 NVL · FP8 · vLLM · tp4 mtp3, aggregate throughput up to 5,245 tok/s on 4x H200 NVL · FP8 · vLLM · tp4 mtp3 and verified context of 968,377 tokens.

Last measured 2026-10-13
NVFP4 (nvidia checkpoint, fp8 KV)NVFP4 (RedHatAI checkpoint, bf16 KV)NVFP4 (RedHatAI checkpoint) + MTP k=2NVFP4FP8IQ4_XSvLLMvLLM, tonyd2wild v11 DFlash2vLLM, tonyd2wild v11 DFlash2 k7llama.cpp VulkanUp to 16 usersVerified context 968,377 tokens38 tool-call tasks passed

Real throughput, latency and accuracy numbers, measured by Petronella Technology Group, Inc. on our own fleet. Last measured 2026-10-13.

Every number traced to a source file|No vendor marketing figures
238 tok/s
best single-user speed
968,377 tokens
verified context
17
measured configurations
Model card

At a glance

Hardware needed1x GB300 (DGX Station GB300, dgx1), 2x GB10, TP2 (msi3+4), 4x GB10, TP4 (msi1-4), 4x H200 NVL (ptg-rtx, TP4), 2x RTX PRO 6000 Blackwell Max-Q (h200monsta), 1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT)
EnginevLLM, vLLM, tonyd2wild v11 DFlash2, vLLM, tonyd2wild v11 DFlash2 k7, llama.cpp Vulkan
QuantizationNVFP4 (nvidia checkpoint, fp8 KV), NVFP4 (RedHatAI checkpoint, bf16 KV), NVFP4 (RedHatAI checkpoint) + MTP k=2, NVFP4, FP8, IQ4_XS
Best single-user speed238 tok/s
Best aggregate (8 users)not measured
Verified context968,377 tokens
The fastest single-user prose speed we measured for this model is 238 tokens per second.

Checked, not assumed

Needle found at 968,377 tokens
1x GB300 · NVFP4 (nvidia checkpoint, fp8 KV)
38 tool-call tasks passed
1x GB300 · NVFP4 (nvidia checkpoint, fp8 KV)
90% strict on our CMMC retrieval battery
4x H200 NVL · FP8 · vLLM · cmmc rag v3

How speed holds up as users are added

Configuration1 user8 users16 users32 usersAt ceiling
1x GB300 · NVFP4 (nvidia checkpoint, fp8 KV)145.9 tok/s1,464 tok/s
1x GB300 · NVFP4 (RedHatAI checkpoint, bf16 KV)144 tok/s1,633 tok/s
1x GB300 · NVFP4 (RedHatAI checkpoint) + MTP k=2231 tok/s1,760 tok/s
2x GB10, TP2 · NVFP424.1 tok/s68.3 tok/s at 6 users
4x GB10, TP4 · NVFP444.9 tok/s258.6 tok/s174.3 tok/s at 16 users
4x H200 NVL · FP8 · vLLM · tp4 mtp3238 tok/s2,254 tok/s
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · aligned166 tok/s394 tok/s430 tok/s
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · aligned all166 tok/s444 tok/s419 tok/s
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · aligned chunk1024159 tok/s379 tok/s402 tok/s
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · aligned chunk2048156 tok/s410 tok/s399 tok/s
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · aligned low152 tok/s406 tok/s395 tok/s
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · aligned mb256164 tok/s425 tok/s414 tok/s
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · vram160 tok/s435 tok/s433 tok/s
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · vram effort low176 tok/s434 tok/s454 tok/s

Aggregate tokens per second across all users at once. One user is a single stream.

Prefill by prompt length

8K prompt32K prompt128K prompttime to first token, 32K
1x GB300 · NVFP4 (RedHatAI checkpoint, bf16 KV)28,610 tok/s33,076 tok/s32,511 tok/s
2x GB10, TP2 · NVFP41,604 tok/s987 tok/s1,171 tok/s
4x GB10, TP4 · NVFP42,120 tok/s2,049 tok/s2,048 tok/s
4x H200 NVL · FP8 · vLLM · tp4 mtp320,763.5 tok/s22,696.7 tok/s21,999.8 tok/s1.444 s

Adding units

ConfigurationSingle userAggregate
2x GB10, TP2 · NVFP424.1 tok/s
4x GB10, TP4 · NVFP444.9 tok/s258.6 tok/s
Measured numbers

Every measured number

Speed (single user)

Configurationdecode 1u prosedecode 1u code
1x GB300 (DGX Station GB300, dgx1) · vLLM · NVFP4 (nvidia checkpoint, fp8 KV)145.9 tok/s
1x GB300 (DGX Station GB300, dgx1) · vLLM · NVFP4 (RedHatAI checkpoint, bf16 KV)144 tok/s
1x GB300 (DGX Station GB300, dgx1) · vLLM · NVFP4 (RedHatAI checkpoint) + MTP k=2231 tok/s
2x GB10, TP2 (msi3+4) · vLLM, tonyd2wild v11 DFlash2 · NVFP424.1 tok/s26.8 tok/s
4x GB10, TP4 (msi1-4) · vLLM, tonyd2wild v11 DFlash2 k7 · NVFP444.9 tok/s49.9 tok/s
4x H200 NVL (ptg-rtx, TP4) · vLLM · FP8238 tok/s
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP8166 tok/s
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP8166 tok/s
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP8159 tok/s
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP8156 tok/s
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP8152 tok/s
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP8164 tok/s
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP8160 tok/s
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP8176 tok/s
1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Vulkan · IQ4_XS15 tok/s

Source: fleet-bench/dgx1-glm53-flash-service.md, fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/h200monsta-glm-recipe-2026-10-10/summary.md, ai11/RESULTS-2026-10-13-model-sweep.md

Aggregate throughput

Configurationagg decode 32uceiling usersagg decode at ceilingpeak agg decode
1x GB300 (DGX Station GB300, dgx1) · vLLM · NVFP4 (nvidia checkpoint, fp8 KV)1,464 tok/s2,517 tok/s
1x GB300 (DGX Station GB300, dgx1) · vLLM · NVFP4 (RedHatAI checkpoint, bf16 KV)1,633 tok/s2,579 tok/s
1x GB300 (DGX Station GB300, dgx1) · vLLM · NVFP4 (RedHatAI checkpoint) + MTP k=21,760 tok/s
2x GB10, TP2 (msi3+4) · vLLM, tonyd2wild v11 DFlash2 · NVFP46 users68.3 tok/s
4x GB10, TP4 (msi1-4) · vLLM, tonyd2wild v11 DFlash2 k7 · NVFP4258.6 tok/s16 users174.3 tok/s
4x H200 NVL (ptg-rtx, TP4) · vLLM · FP82,254 tok/s5,245 tok/s

Source: fleet-bench/dgx1-glm53-flash-service.md, fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-10-02-best-model-per-tier-results.md

Prefill and latency

Configurationprefill 8kprefill 32kprefill 128kttft 32k
1x GB300 (DGX Station GB300, dgx1) · vLLM · NVFP4 (RedHatAI checkpoint, bf16 KV)28,610 tok/s33,076 tok/s32,511 tok/s
2x GB10, TP2 (msi3+4) · vLLM, tonyd2wild v11 DFlash2 · NVFP41,604 tok/s987 tok/s1,171 tok/s
4x GB10, TP4 (msi1-4) · vLLM, tonyd2wild v11 DFlash2 k7 · NVFP42,120 tok/s2,049 tok/s2,048 tok/s
4x H200 NVL (ptg-rtx, TP4) · vLLM · FP820,763.5 tok/s22,696.7 tok/s21,999.8 tok/s1.444 s

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-09-25-dgx1-eval-glm53-flash-fp8-h200.json

Context

Configurationcontext verified
1x GB300 (DGX Station GB300, dgx1) · vLLM · NVFP4 (nvidia checkpoint, fp8 KV)968,377 tokens
4x H200 NVL (ptg-rtx, TP4) · vLLM · FP8510,997 tokens

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md

Quality

Configurationcmmc rag strict pcttoolcall pass
1x GB300 (DGX Station GB300, dgx1) · vLLM · NVFP4 (nvidia checkpoint, fp8 KV)38 count
4x H200 NVL (ptg-rtx, TP4) · vLLM · FP890 %
4x H200 NVL (ptg-rtx, TP4) · vLLM · FP80.964 %
4x H200 NVL (ptg-rtx, TP4) · vLLM · FP823 count

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-10-03-cmmc-rag-v3-results.md, fleet-bench/2026-09-25-dgx1-eval-glm53-flash-fp8-h200.json

Power and efficiency

Configurationtok per joule
2x GB10, TP2 (msi3+4) · vLLM, tonyd2wild v11 DFlash2 · NVFP40.23 tok/J
4x GB10, TP4 (msi1-4) · vLLM, tonyd2wild v11 DFlash2 k7 · NVFP40.45 tok/J

Source: fleet-bench/2026-10-02-best-model-per-tier-results.md

Other measurements

ConfigurationClosed-book CMMC items correct of 35 (vendor sampling)HumanEval pass@1 (vendor sampling)CMMC RAG v3 frozen-passage strict score, seed 21CMMC RAG v3 closed-book score, seed 21CMMC RAG v3 code-first strict, mean over 3 seedsCMMC RAG v3 frozen strict, mean over 3 seedsCMMC RAG v3 closed-book strict, 1 seedTool-call v2 overall accuracyAgentic tool completion rate (12 cases)Tool-call hallucination rateKV cache capacity from /metricsEval suite total passed (tools 38, agent 15, cmmc 28, code 10, write 3, long 26)Sam first-token latency, new turnSam turn total, medianAggregate decode at 8 streamsAggregate decode at 16 streamsGLM corpus score of 264prefill speedMMLU-Pro correct of 40 (60-item battery)HumanEval correct of 20 (60-item battery)
1x GB300 (DGX Station GB300, dgx1) · vLLM · NVFP4 (nvidia checkpoint, fp8 KV)26 count
1x GB300 (DGX Station GB300, dgx1) · vLLM · NVFP4 (RedHatAI checkpoint, bf16 KV)26 count97 %
4x H200 NVL (ptg-rtx, TP4) · vLLM · FP883 count45 count
4x H200 NVL (ptg-rtx, TP4) · vLLM · FP80.928 rate0.892 rate0.484 rate
4x H200 NVL (ptg-rtx, TP4) · vLLM · FP885.2 %0.917 rate0 %3,329,280 tokens120 count
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP80.37 s0.53 s394 tok/s430 tok/s224 count
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP80.34 s0.47 s444 tok/s419 tok/s221 count
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP80.58 s0.68 s379 tok/s402 tok/s221 count
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP80.46 s0.59 s410 tok/s399 tok/s227 count
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP80.39 s0.47 s406 tok/s395 tok/s215 count
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP80.4 s0.55 s425 tok/s414 tok/s224 count
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP80.14 s2.37 s435 tok/s433 tok/s227 count
2x RTX PRO 6000 Blackwell Max-Q (h200monsta) · vLLM · FP80.14 s2.32 s434 tok/s454 tok/s221 count
1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Vulkan · IQ4_XS173 tok/s36 of 4019 of 20

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-10-03-cmmc-rag-v3-results.md, fleet-bench/2026-09-25-dgx1-eval-glm53-flash-fp8-h200.json, fleet-bench/h200monsta-glm-recipe-2026-10-10/summary.md, ai11/RESULTS-2026-10-13-model-sweep.md

Review

GLM-5.3-Flash is the large mixture-of-experts model our team keeps coming back to when the job needs both reasoning depth and a very long context window on hardware we own. We measured it on four very different machines, from a desk-side GB10 pod to the four-GPU H200 server that runs our production assistant, and this page reports every number we recorded, including the configurations where it is slow or failed outright.

What it is

GLM-5.3-Flash is Z.ai's large mixture-of-experts model. "Flash" in the name signals the speed-oriented variant of the GLM family, and in our fleet it is the model we reach for when a workload needs careful reading of long documents, tool calling that does not hallucinate, and compliance-domain recall, all served from our own racks.

We measured it in several checkpoint formats: an FP8 build on the H200 server, NVFP4 builds on the GB300 workstation and on GB10 pods, and an IQ4_XS llama.cpp build on the AMD Strix Halo mini PC. The same model behaves very differently across those formats, so the configuration matters as much as the model choice.

Two qualities set it apart in our measurements. First, its tool calling is disciplined: on the H200 production build it passed all eval tool cases with a zero percent hallucination rate, and the agentic completion rate came in at 0.917 with fabrication kept low. Second, it verified a needle at 510,997 tokens of context on the H200 build and passed a full-context correctness battery at 968,377 tokens on the GB300. If your use case involves whole-contract or whole-codebase prompts, those verified numbers are the ones to compare against.

Where it runs best

The honest answer is that GLM-5.3-Flash wants real GPU memory bandwidth and many units. Here is how the configurations compared.

Four H200 NVL GPUs (our production server). This is where the model shines. Serving FP8 with tensor parallelism across four cards and multi-token prediction enabled, it decodes at 238 tok/s for a single user and 2,254 tok/s aggregate at 32 users, peaking at 5,245 tok/s at its highest measured concurrency. Prefill is strong and flat across lengths: 20,763.5 tok/s at 8k prompts, 22,696.7 tok/s at 32k, and 21,999.8 tok/s at 128k, with a 32k time-to-first-token of 1.444 seconds. The KV cache on that deployment holds 3,329,280 tokens. This is the configuration we would pick for any team that can afford a four-card server.

DGX Station GB300 (single GB300). The NVFP4 build with the nvidia checkpoint served one user at 145.9 tok/s and 1,464 tok/s aggregate at 32 users, peaking at 2,517 tok/s at its highest measured concurrency, and it passed the long-context battery three of three times at 968,377 tokens. The RedHatAI NVFP4 checkpoint measured almost identically for single-user speed at 144 tok/s, with 1,633 tok/s at 32 users and a peak of 2,579 tok/s at its highest measured concurrency, plus strong prefill of 28,610 / 33,076 / 32,511 tok/s at 8k/32k/128k. A tempting variant loads multi-token prediction on the RedHatAI checkpoint: it hit 231 tok/s for one user and 1,760 at 32 users, but it loses measurable closed-book compliance recall, so we would not deploy it for accuracy-sensitive work. Note also that on the nvidia checkpoint, multi-token prediction does not load in this vLLM build at all.

Four GB10 units (a DGX Spark pod). With tensor parallelism over four units running the DFlash2 speculative recipe, the model serves one user at 44.9 tok/s prose and 49.9 tok/s code, 258.6 tok/s aggregate at 32 users, and holds a ceiling of 16 concurrent users at 174.3 tok/s. Prefill holds steady near 2,120 / 2,049 / 2,048 tok/s across 8k/32k/128k prompts, and it serves a verified million-token context. This is the sweet spot for a quiet office pod that still needs the full long-context GLM experience. Energy efficiency measured 0.45 tok/J, roughly double the two-unit figure.

Two GB10 units. This is where the model starts to strain. Tensor-parallel over two units, it fits only with six sequences and a small pinned KV pool, so single-user prose decoding falls to 24.1 tok/s (26.8 for code) and the aggregate ceiling is 68.3 tok/s across those six users. Prefill was erratic at this size, 1,604 / 987 / 1,171 tok/s across the three lengths, and our own source notes flag the 32k figure as an outlier to treat as noise, since GLM prefill on GB10 has been erratic across runs. Quality on the pod was actually the best we measured anywhere, but at two units the speed and concurrency are simply poor for this model. Our long-context probing on a GB10 pair also hit hard limits: the model starts at well over half a million tokens before running out of memory, and in practice the pair behaves like a much shorter context server with few concurrent requests. Long-context GLM belongs on four units or more.

One AMD Strix Halo mini PC. We ran an IQ4_XS llama.cpp build on the Minisforum box as a provisioning test. It scored well on the quality battery but decoded at only 15 tok/s with 173 tok/s prefill, which is too slow for interactive work on prompts of any size. We list it here so nobody repeats the experiment expecting different results.

Eight GB10 units. We attempted it and it failed to start in both engine states we tried; every rank died during initialization. We do not recommend an eight-unit GLM deployment on GB10 pods.

Measured speed

Single-user decode speed by configuration, prose unless noted:

| Configuration | Decode, 1 user | Aggregate, 32 users | Peak aggregate | |---|---|---|---| | 4x H200, FP8 (production) | 238 | 2,254 | 5,245 | | 1x GB300, NVFP4 (nvidia) | 145.9 | 1,464 | 2,517 | | 1x GB300, NVFP4 (RedHatAI) | 144 | 1,633 | 2,579 | | 1x GB300, NVFP4 + MTP | 231 | 1,760 | not measured | | 4x GB10, NVFP4 | 44.9 | 258.6 | not measured | | 2x GB10, NVFP4 | 24.1 | not measured | not measured | | 1x Strix Halo, IQ4_XS | 15 | not measured | not measured |

The two-unit GB10 numbers deserve emphasis: 24.1 tok/s single-user is usable for one person thinking out loud, but six queued users would each see the service crawl. The four-unit pod is the smallest GB10 configuration we consider viable for this model.

On the H200 server we also measured time-to-first-token for a 32k prompt at 1.444 seconds, and prefill throughput that barely degrades between 32k and 128k prompts. That flat prefill curve matters for long-document work.

On a two-GPU slice of our four-GPU RTX PRO six-thousand Blackwell Max-Q server we also swept serving recipes for a voice-agent workload. Single-stream Sam-turn decode landed between 152 and 176 tok/s depending on the recipe, first-token latency ranged from 0.14 to 0.4 seconds, and median turn completion ranged from 0.47 to 2.37 seconds. Aggregate decode at eight streams landed between 379 and 444 tok/s. Several aggressive recipes in that sweep FAILED outright, including the ones that pushed larger batches, which is a useful reminder that recipe tuning on this model can cost you a working endpoint if you chase the last few percent.

Quality and accuracy

GLM-5.3-Flash is the strongest closed-book compliance model we have measured on our fleet, and it is at or near the top of every quality battery we ran on the hardware where it fits properly.

On the H200 production build, the eval suite totals came to 120 items passed across tool calling, agentic work, compliance closed-book, code, writing and long-context checks. Tool calling passed 23 of 27 v2 cases with 85.2 percent overall accuracy and a 0.0 hallucination rate. The agentic completion rate was 0.917 with fabrication held to a small minority of cases. HumanEval pass@1 measured 97.0 on the GB300 NVFP4 build. Long-context retrieval is not just claimed in the config: the needle test passed at 510,997 tokens on H200, and the GB300 build passed the full-length battery three of three at 968,377 tokens.

Compliance-domain performance is the standout. On the closed-book CMMC battery the model scored 26 of 35 with vendor sampling on both GB300 checkpoints, the best result of any model we measured on that battery. On the newer retrieval-grounded compliance benchmark, code-first strict accuracy measured 0.964 averaged over three seeds, with frozen-passage strict at 0.892. Closed-book strict on the same benchmark measured 0.484, which is a useful honesty check: with documents in context this model is excellent, from memory alone it is mediocre, exactly like every other model we tested.

Two caveats. The RedHatAI checkpoint with multi-token prediction enabled loses measurable closed-book compliance recall despite being the fastest GB300 variant, so treat speed-ups that touch the decode path as accuracy risks until retested. And the Strix Halo IQ4_XS build, while scoring well on the small battery, was provisioned only as a first pass at greedy temperature, so we treat its quality numbers as provisional.

How we ran it

All measurements ran on our own hardware, with the engine and checkpoint recorded per configuration. The H200 production service runs vLLM with FP8 weights, tensor parallelism across four cards and multi-token prediction, and we measured it read-only while it was live. The GB300 runs use vLLM with NVFP4 checkpoints, one from nvidia and one from RedHatAI, and we measured the deployed service configuration separately from the eval comparison configuration. The GB10 pod runs use a vLLM fork with the DFlash2 speculative decoding recipe at a fixed clock. The Strix Halo run uses llama.cpp with Vulkan.

Concurrency numbers come from ladder runs that step the request count and measure aggregate decode at each step. Quality batteries mix vendor sampling with our own harness, and every battery row on this page names the source file it came from. Where a source flagged a result as invalid, retracted or an outlier, we left it out of the tables or marked it in the text. We did not correct, round or extrapolate any number: each value on this page is the value in the record.

When to pick it

Pick GLM-5.3-Flash when accuracy on compliance and long documents matters more than tokens per joule, and when your hardware has the memory to host it properly. Concretely:

  • Four H200-class GPUs: this is its best home, with the highest single-user and peak throughput and proven long-context behavior. It is what we run in production.
  • A GB300 workstation: nearly as good, with the caveat that the multi-token-predictor variant trades compliance recall for speed.
  • A four-unit GB10 pod: the right choice when noise, power and desk-side placement matter and you still need million-token context. Expect about 45 tok/s per user and a 16-user ceiling.
  • Anything smaller: choose a different model. On two GB10 units or a single Strix Halo mini PC this model is too slow to be pleasant, and on eight GB10 units it would not start for us at all.

If your workload is mostly short prompts with light concurrency and you want maximum efficiency instead, other models in our benchmark suite will serve you better. But for private deployment where the model must read a full contract, a full codebase, or a full compliance evidence package in one prompt and then answer tool-driven questions without fabricating, GLM-5.3-Flash is the model our own fleet settles on.

How each configuration was served

ConfigurationHardwareUnitsEngineQuantMeasuredNotes
1x GB300 · NVFP4 (nvidia checkpoint, fp8 KV)1x GB300 (DGX Station GB300, dgx1)1vLLMNVFP4 (nvidia checkpoint, fp8 KV)2026-09-25
1x GB300 · NVFP4 (RedHatAI checkpoint, bf16 KV)1x GB300 (DGX Station GB300, dgx1)1vLLMNVFP4 (RedHatAI checkpoint, bf16 KV)2026-09-25Eval comparison quant. Superseded for deployment by the nvidia checkpoint.
1x GB300 · NVFP4 (RedHatAI checkpoint) + MTP k=21x GB300 (DGX Station GB300, dgx1)1vLLMNVFP4 (RedHatAI checkpoint) + MTP k=22026-09-25
2x GB10, TP2 · NVFP42x GB10, TP2 (msi3+4)2vLLM, tonyd2wild v11 DFlash2NVFP42026-10-02Best quality on the pod (CMMC 26 of 35 closed-book, 0% hallucinated tool calls) but at TP2 it fits only with 6 sequences and 24-28 tok/s; long-context GLM belongs on 4 or more units. The 32k prefill value of 987 is flagged in the source as an outlier to treat as noise.
4x GB10, TP4 · NVFP44x GB10, TP4 (msi1-4)4vLLM, tonyd2wild v11 DFlash2 k7NVFP42026-10-02
4x H200 NVL · FP8 · vLLM · cmmc rag v34x H200 NVL (ptg-rtx, TP4)4vLLMFP82026-10-02Reference rows from the v3 builder (ptg-rtx/H200, seed 21, /92 before item mh_poam_3_10_4).
4x H200 NVL · FP8 · vLLM · cmmc rag v3 ladder4x H200 NVL (ptg-rtx, TP4)4vLLMFP82026-10-03CMMC-RAG v3 ladder row; GLM-5.3-Flash FP8 served from the ptg-rtx 4x H200 production endpoint.
4x H200 NVL · FP8 · vLLM · tp4 mtp34x H200 NVL (ptg-rtx, TP4)4vLLMFP82026-09-25
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · aligned2x RTX PRO 6000 Blackwell Max-Q (h200monsta)2vLLMFP82026-10-10
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · aligned all2x RTX PRO 6000 Blackwell Max-Q (h200monsta)2vLLMFP82026-10-10
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · aligned chunk10242x RTX PRO 6000 Blackwell Max-Q (h200monsta)2vLLMFP82026-10-10
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · aligned chunk20482x RTX PRO 6000 Blackwell Max-Q (h200monsta)2vLLMFP82026-10-10
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · aligned low2x RTX PRO 6000 Blackwell Max-Q (h200monsta)2vLLMFP82026-10-10
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · aligned mb2562x RTX PRO 6000 Blackwell Max-Q (h200monsta)2vLLMFP82026-10-10
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · vram2x RTX PRO 6000 Blackwell Max-Q (h200monsta)2vLLMFP82026-10-10
2x RTX PRO 6000 Blackwell Max-Q · FP8 · vLLM · vram effort low2x RTX PRO 6000 Blackwell Max-Q (h200monsta)2vLLMFP82026-10-10
1x ai11 · IQ4_XS1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT)1llama.cpp VulkanIQ4_XS2026-10-13Provisional first pass on one engine at greedy temp 0; a Gufo fairness rerun was queued.

FAQ

How fast is GLM-5.3-Flash on four H200 GPUs?
On our four-card FP8 production build it decodes at 238 tok/s for one user with multi-token prediction enabled, 2,254 tok/s aggregate at 32 users, and 5,245 tok/s peak at its highest measured concurrency. Prefill barely degrades even at 128k-token prompts.
Is it usable on a single GB10 DGX Spark?
Not as tensor-parallel single unit in our tests, and on a two-unit pod it measured only 24.1 tok/s of single-user prose decoding with a six-user ceiling at 68.3 tok/s aggregate. We recommend at least four GB10 units for this model, where it reaches 44.9 tok/s single-user.
Did it pass a long-context correctness test?
Yes. On the GB300 NVFP4 build it passed the battery three of three times at 968,377 tokens of prompt. On H200 it retrieved a needle placed at 510,997 tokens end to end in well under half a minute, and the production deployment serves a verified half-million-token window.
Does multi-token prediction help?
It helps speed and can cost accuracy. On the GB300 RedHatAI checkpoint, enabling it raised single-user decode from 144 to 231 tok/s and 32-user aggregate from 1,633 to 1,760, but the model lost measurable closed-book compliance recall. We do not enable it where accuracy matters.
How is its tool calling?
Excellent and honest. On H200 it passed 23 of 27 tool-call v2 cases at 85.2 percent accuracy with a 0.0 hallucination rate, and on the GB300 it passed all 38 eval tool cases. Its agentic completion rate measured 0.917 with fabrication on only a small minority of cases.
How does it do on compliance questions without documents in context?
Mediocre, like every model. Closed-book strict accuracy on our retrieval-grounded compliance benchmark measured 0.484, while the same model with code-first retrieval scored 0.964. Serve it with retrieval; do not trust it from memory alone.
What happened on eight GB10 units?
It failed to start. In both engine states we tried, every tensor-parallel rank died during startup. We keep the logs with the failure and do not recommend an eight-unit GB10 GLM deployment.
Can it run on a small mini PC?
We ran an IQ4_XS llama.cpp build on one AMD Strix Halo box. Quality on the short battery was strong, but 15 tok/s decode and 173 tok/s prefill make interactive long-prompt work impractical. Treat mini PCs as provisioning tests for this model, not deployment targets.
What checkpoint should I serve?
On H200, the FP8 checkpoint with multi-token prediction is our production choice. On GB300, the nvidia NVFP4 checkpoint is the deployed service, while the RedHatAI checkpoint measures nearly the same speed with slightly different aggregate behavior. On GB10 pods, NVFP4 with the DFlash2 recipe is the only configuration we got to serve reliably.
How energy efficient is it?
On the GB10 pod, efficiency improved from 0.23 tok/J on two units to 0.45 tok/J on four units, so adding units made the pod more efficient per token, not less. We did not measure joules on the server-class hardware.

Run this model on your own hardware

Call Penny at 919-348-4912 and describe your fleet, or ask us to benchmark a model on the hardware you already own.