GLM-5.3-Flashbenchmark results

Measured speed and quality

Speed (single user)

Source: fleet-bench/dgx1-glm53-flash-service.md, fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/h200monsta-glm-recipe-2026-10-10/summary.md, ai11/RESULTS-2026-10-13-model-sweep.md

Aggregate throughput

Source: fleet-bench/dgx1-glm53-flash-service.md, fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-10-02-best-model-per-tier-results.md

Prefill and latency

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-09-25-dgx1-eval-glm53-flash-fp8-h200.json

Context

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md

Quality

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-10-03-cmmc-rag-v3-results.md, fleet-bench/2026-09-25-dgx1-eval-glm53-flash-fp8-h200.json

Power and efficiency

Source: fleet-bench/2026-10-02-best-model-per-tier-results.md

Other measurements

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/h200monsta-glm-recipe-2026-10-10/summary.md, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-10-03-cmmc-rag-v3-results.md, fleet-bench/2026-09-25-dgx1-eval-glm53-flash-fp8-h200.json, ai11/RESULTS-2026-10-13-model-sweep.md

GLM-5.3-Flash is the large mixture-of-experts model our team keeps coming back to when the job needs both reasoning depth and a very long context window on hardware we own. We measured it on four very different machines, from a desk-side GB10 pod to the four-GPU H200 server that runs our production assistant, and this page reports every number we recorded, including the configurations where it is slow or failed outright.

What it is

GLM-5.3-Flash is Z.ai's large mixture-of-experts model. "Flash" in the name signals the speed-oriented variant of the GLM family, and in our fleet it is the model we reach for when a workload needs careful reading of long documents, tool calling that does not hallucinate, and compliance-domain recall, all served from our own racks.

We measured it in several checkpoint formats: an FP8 build on the H200 server, NVFP4 builds on the GB300 workstation and on GB10 pods, and an IQ4_XS llama.cpp build on the AMD Strix Halo mini PC. The same model behaves very differently across those formats, so the configuration matters as much as the model choice.

Two qualities set it apart in our measurements. First, its tool calling is disciplined: on the H200 production build it passed all eval tool cases with a zero percent hallucination rate, and the agentic completion rate came in at 0.917 with fabrication kept low. Second, it verified a needle at 510,997 tokens of context on the H200 build and passed a full-context correctness battery at 968,377 tokens on the GB300. If your use case involves whole-contract or whole-codebase prompts, those verified numbers are the ones to compare against.

Where it runs best

The honest answer is that GLM-5.3-Flash wants real GPU memory bandwidth and many units. Here is how the configurations compared.

Four H200 NVL GPUs (our production server). This is where the model shines. Serving FP8 with tensor parallelism across four cards and multi-token prediction enabled, it decodes at 238 tok/s for a single user and 2,254 tok/s aggregate at 32 users, peaking at 5,245 tok/s at its highest measured concurrency. Prefill is strong and flat across lengths: 20,763.5 tok/s at 8k prompts, 22,696.7 tok/s at 32k, and 21,999.8 tok/s at 128k, with a 32k time-to-first-token of 1.444 seconds. The KV cache on that deployment holds 3,329,280 tokens. This is the configuration we would pick for any team that can afford a four-card server.

DGX Station GB300 (single GB300). The NVFP4 build with the nvidia checkpoint served one user at 145.9 tok/s and 1,464 tok/s aggregate at 32 users, peaking at 2,517 tok/s at its highest measured concurrency, and it passed the long-context battery three of three times at 968,377 tokens. The RedHatAI NVFP4 checkpoint measured almost identically for single-user speed at 144 tok/s, with 1,633 tok/s at 32 users and a peak of 2,579 tok/s at its highest measured concurrency, plus strong prefill of 28,610 / 33,076 / 32,511 tok/s at 8k/32k/128k. A tempting variant loads multi-token prediction on the RedHatAI checkpoint: it hit 231 tok/s for one user and 1,760 at 32 users, but it loses measurable closed-book compliance recall, so we would not deploy it for accuracy-sensitive work. Note also that on the nvidia checkpoint, multi-token prediction does not load in this vLLM build at all.

Four GB10 units (a DGX Spark pod). With tensor parallelism over four units running the DFlash2 speculative recipe, the model serves one user at 44.9 tok/s prose and 49.9 tok/s code, 258.6 tok/s aggregate at 32 users, and holds a ceiling of 16 concurrent users at 174.3 tok/s. Prefill holds steady near 2,120 / 2,049 / 2,048 tok/s across 8k/32k/128k prompts, and it serves a verified million-token context. This is the sweet spot for a quiet office pod that still needs the full long-context GLM experience. Energy efficiency measured 0.45 tok/J, roughly double the two-unit figure.

Two GB10 units. This is where the model starts to strain. Tensor-parallel over two units, it fits only with six sequences and a small pinned KV pool, so single-user prose decoding falls to 24.1 tok/s (26.8 for code) and the aggregate ceiling is 68.3 tok/s across those six users. Prefill was erratic at this size, 1,604 / 987 / 1,171 tok/s across the three lengths, and our own source notes flag the 32k figure as an outlier to treat as noise, since GLM prefill on GB10 has been erratic across runs. Quality on the pod was actually the best we measured anywhere, but at two units the speed and concurrency are simply poor for this model. Our long-context probing on a GB10 pair also hit hard limits: the model starts at well over half a million tokens before running out of memory, and in practice the pair behaves like a much shorter context server with few concurrent requests. Long-context GLM belongs on four units or more.

One AMD Strix Halo mini PC. We ran an IQ4_XS llama.cpp build on the Minisforum box as a provisioning test. It scored well on the quality battery but decoded at only 15 tok/s with 173 tok/s prefill, which is too slow for interactive work on prompts of any size. We list it here so nobody repeats the experiment expecting different results.

Eight GB10 units. We attempted it and it failed to start in both engine states we tried; every rank died during initialization. We do not recommend an eight-unit GLM deployment on GB10 pods.

Measured speed

Single-user decode speed by configuration, prose unless noted:

| Configuration | Decode, 1 user | Aggregate, 32 users | Peak aggregate | |---|---|---|---| | 4x H200, FP8 (production) | 238 | 2,254 | 5,245 | | 1x GB300, NVFP4 (nvidia) | 145.9 | 1,464 | 2,517 | | 1x GB300, NVFP4 (RedHatAI) | 144 | 1,633 | 2,579 | | 1x GB300, NVFP4 + MTP | 231 | 1,760 | not measured | | 4x GB10, NVFP4 | 44.9 | 258.6 | not measured | | 2x GB10, NVFP4 | 24.1 | not measured | not measured | | 1x Strix Halo, IQ4_XS | 15 | not measured | not measured |

The two-unit GB10 numbers deserve emphasis: 24.1 tok/s single-user is usable for one person thinking out loud, but six queued users would each see the service crawl. The four-unit pod is the smallest GB10 configuration we consider viable for this model.

On the H200 server we also measured time-to-first-token for a 32k prompt at 1.444 seconds, and prefill throughput that barely degrades between 32k and 128k prompts. That flat prefill curve matters for long-document work.

On a two-GPU slice of the H200 server we also swept serving recipes for a voice-agent workload. Single-stream Sam-turn decode landed between 152 and 176 tok/s depending on the recipe, first-token latency ranged from 0.14 to 0.4 seconds, and median turn completion ranged from 0.47 to 2.37 seconds. Aggregate decode at eight streams landed between 379 and 444 tok/s. Several aggressive recipes in that sweep FAILED outright, including the ones that pushed larger batches, which is a useful reminder that recipe tuning on this model can cost you a working endpoint if you chase the last few percent.

Quality and accuracy

GLM-5.3-Flash is the strongest closed-book compliance model we have measured on our fleet, and it is at or near the top of every quality battery we ran on the hardware where it fits properly.

On the H200 production build, the eval suite totals came to 120 items passed across tool calling, agentic work, compliance closed-book, code, writing and long-context checks. Tool calling passed 23 of 27 v2 cases with 85.2 percent overall accuracy and a 0.0 hallucination rate. The agentic completion rate was 0.917 with fabrication held to a small minority of cases. HumanEval pass@1 measured 97.0 on the GB300 NVFP4 build. Long-context retrieval is not just claimed in the config: the needle test passed at 510,997 tokens on H200, and the GB300 build passed the full-length battery three of three at 968,377 tokens.

Compliance-domain performance is the standout. On the closed-book CMMC battery the model scored 26 of 35 with vendor sampling on both GB300 checkpoints, the best result of any model we measured on that battery. On the newer retrieval-grounded compliance benchmark, code-first strict accuracy measured 0.964 averaged over three seeds, with frozen-passage strict at 0.892. Closed-book strict on the same benchmark measured 0.484, which is a useful honesty check: with documents in context this model is excellent, from memory alone it is mediocre, exactly like every other model we tested.

Two caveats. The RedHatAI checkpoint with multi-token prediction enabled loses measurable closed-book compliance recall despite being the fastest GB300 variant, so treat speed-ups that touch the decode path as accuracy risks until retested. And the Strix Halo IQ4_XS build, while scoring well on the small battery, was provisioned only as a first pass at greedy temperature, so we treat its quality numbers as provisional.

How we ran it

All measurements ran on our own hardware, with the engine and checkpoint recorded per configuration. The H200 production service runs vLLM with FP8 weights, tensor parallelism across four cards and multi-token prediction, and we measured it read-only while it was live. The GB300 runs use vLLM with NVFP4 checkpoints, one from nvidia and one from RedHatAI, and we measured the deployed service configuration separately from the eval comparison configuration. The GB10 pod runs use a vLLM fork with the DFlash2 speculative decoding recipe at a fixed clock. The Strix Halo run uses llama.cpp with Vulkan.

Concurrency numbers come from ladder runs that step the request count and measure aggregate decode at each step. Quality batteries mix vendor sampling with our own harness, and every battery row on this page names the source file it came from. Where a source flagged a result as invalid, retracted or an outlier, we left it out of the tables or marked it in the text. We did not correct, round or extrapolate any number: each value on this page is the value in the record.

When to pick it

Pick GLM-5.3-Flash when accuracy on compliance and long documents matters more than tokens per joule, and when your hardware has the memory to host it properly. Concretely:

• Four H200-class GPUs: this is its best home, with the highest single-user and peak throughput and proven long-context behavior. It is what we run in production.
• A GB300 workstation: nearly as good, with the caveat that the multi-token-predictor variant trades compliance recall for speed.
• A four-unit GB10 pod: the right choice when noise, power and desk-side placement matter and you still need million-token context. Expect about 45 tok/s per user and a 16-user ceiling.
• Anything smaller: choose a different model. On two GB10 units or a single Strix Halo mini PC this model is too slow to be pleasant, and on eight GB10 units it would not start for us at all.

If your workload is mostly short prompts with light concurrency and you want maximum efficiency instead, other models in our benchmark suite will serve you better. But for private deployment where the model must read a full contract, a full codebase, or a full compliance evidence package in one prompt and then answer tool-driven questions without fabricating, GLM-5.3-Flash is the model our own fleet settles on.

Where to go next

• LLM benchmarks hub
• Leaderboards
• How we measure
• Model tier list
• Private AI solutions
• Private LLM guide
• CMMC compliance

FAQ