Qwen3.8-27Bbenchmark results

Measured speed and quality

Speed (single user)

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-09-23-dgx1-dense-qwen38-27b-fp8.json, fleet-bench/2026-09-23-dgx1-dense-qwen38-27b-nvfp4.json, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-09-23-h200monsta-qwen38-27b-concurrency.json

Aggregate throughput

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-09-23-dgx1-dense-qwen38-27b-fp8.json, fleet-bench/2026-09-23-dgx1-dense-qwen38-27b-nvfp4.json, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-09-21-h200monsta-qwen38-27b-seo-shape-seqs32.json, fleet-bench/2026-09-21-h200monsta-qwen38-27b-seo-shape-seqs64-bt32768.json, fleet-bench/2026-09-23-h200monsta-qwen38-27b-concurrency.json

Prefill and latency

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md

Context

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-09-23-h200monsta-qwen38-27b-concurrency.json

Quality

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-10-03-cmmc-rag-v3-results.md, fleet-bench/2026-10-02-best-model-per-tier-results.md

Power and efficiency

Source: fleet-bench/2026-10-02-best-model-per-tier-results.md

Other measurements

Source: fleet-bench/2026-09-25-dgx1-best-model-eval.md, fleet-bench/2026-09-28-dgx1-huihui-qwen38-abliterated.md, fleet-bench/2026-10-03-cmmc-rag-v3-results.md, fleet-bench/2026-09-21-h200monsta-qwen38-27b-seo-shape-seqs32.json, fleet-bench/2026-09-21-h200monsta-qwen38-27b-seo-shape-seqs64-bt32768.json, fleet-bench/2026-09-23-h200monsta-qwen38-27b-concurrency.json, ai11/RESULTS-2026-10-13-model-sweep.md

What it is

Qwen3.8-27B is a dense 27-billion-parameter model from Alibaba's Qwen family. Unlike the mixture-of-experts models in this benchmark set, every one of its parameters runs on every token, which makes it heavier to serve than a same-class MoE model but steadier to reason about: there is no expert-routing lottery, and its behavior does not change when you scale the number of units. We use it internally as the dense reference point whenever we score a new MoE model on the fleet, and that role is why it has more measured configurations than almost anything else on these pages.

The headline finding from our fleet data is simple and a little surprising: with retrieval wired in, this model punches at the level of models several times its size, and it is the best tool-calling and long-context dense model we have measured on a single GB10 unit. Without retrieval, its compliance recall is genuinely weak. That split, good when grounded, weak closed-book, is the single most important thing to understand about this model before you deploy it.

The family name appears in our records under several checkpoints: the stock Qwen3.8-27B in NVFP4, FP8 and BF16 quantizations, a DavidAU community fusion build in Q6_K that ran on the AMD Strix Halo box, and an abliterated Huihui variant we serve for authorized red-team work only. All of them are the same architecture, so the speed and quality pattern carries across the quantizations even though the absolute numbers differ.

Where it runs best

The records cover four hardware families, and the honest ranking for this model looks like this.

DGX Station GB300 (dgx1) is where it is fastest. Single-user prose decode measured at 162.3 tok/s with the NVFP4 quantization, 111.2 tok/s with FP8, and 90 tok/s in full BF16. Aggregate throughput at 32 concurrent users reached 3,552.3 tok/s on NVFP4 and 2,510.5 tok/s on FP8, and the BF16 configuration scaled to a peak aggregate of 4,278 tok/s at the top of the ladder. If you have a Station-class box, this model is fast enough to serve a whole team from one card, and the NVFP4 checkpoint is the fastest of the three quants by a clear margin.

H200 (one card) is the solid mid-tier. The FP8 build on a a single card decoded at 80.1 tok/s for one user and peaked at 1,358.0 tok/s aggregate at 32 users, holding 32 users at the ceiling before the engine's sequence cap queued anything. On the real SEO-shaped workload, an 8,646-token prompt with long outputs, the same service managed 264.4 tok/s aggregate at 8 users, and pushing to 64 users with a larger block size only reached 270.9 tok/s at the cost of minute-scale time to first token. Read that as: short prompts are this model's home turf, and very long prompts on a single card will make callers wait.

A single GB10 unit is where it is slow, and this is the honest weak spot. On one DGX Spark-class unit it decoded at 24.4 tok/s for prose and 32.3 tok/s for code, reaching 316.6 tok/s aggregate at 32 users. That is roughly a quarter of the speed of the fastest MoE models on the same hardware, which exchanged about four times Nemotron's speed for the best quality in the one-unit tier. It also passed 7 of 8 gate runs plus a recheck, so one gate item needed a second look. If you need interactive chat on one Spark unit, pick a Flash-class model; this one is the careful, patient option.

AMD Strix Halo (ai11) is the slowest measured configuration. The DavidAU fusion build in Q6_K generated at 15 to 22 tok/s with prefill of 162 tok/s, but it posted the highest quality score in that sweep: 38 of 40 on the MMLU-Pro battery and 20 of 20 on HumanEval. It is a patient-jobs model there, best for overnight or background work where correctness outranks response time.

The RTX PRO 2000 (16 GB) deployment is a special case. The abliterated Huihui variant in IQ3_S with MTP speculative decoding served 4 concurrent slots at 87.8 tok/s aggregate with prefill of 600 tok/s, and prefilled a 28,305-token prompt at 496 tok/s without running out of memory. That is a purpose-built deployment for the security team's authorized offensive-testing work on our own infrastructure, never a general-purpose alias, and it is included here because it shows the model family fits a 16 GB card when quantized hard.

Measured speed

The verified numbers, configuration by configuration:

• DGX Station GB300, NVFP4, vLLM: 162.3 tok/s single-user prose decode, 3,552.3 tok/s aggregate at 32 users.

Fastest single-stream of the three quants we measured on this box.

• DGX Station GB300, FP8 e4m3 dynamic, vLLM: 111.2 tok/s single-user, 2,510.5 tok/s aggregate at 32 users.
• DGX Station GB300, BF16 with bf16 KV cache, vLLM: 90 tok/s single-user prose, 1,842 tok/s aggregate at 32

users, peaking at 4,278 tok/s at the top of the ladder. The ceiling was not reached because the engine's max-num-seqs capped by the engine, not by the model. Prefill measured 24,558 tok/s at 8k prompts, 24,215 tok/s at 32k and 18,411 tok/s at 128k, which makes long-document ingestion one of this model's real strengths on the GB300.

• One GB10 (DGX Spark class), NVFP4, vLLM nightly with MTP3: 24.4 tok/s prose and 32.3 tok/s code single-user,

316.6 tok/s aggregate at 32 users, which is also the ceiling. Efficiency measured 2.39 tok/J at the ceiling.

• One card of the four H200-class box (the others were busy), FP8, vLLM with TP=1: 80.1 tok/s for one

user, 568.8 tok/s aggregate at 8 users, 943.2 at 16, 1,358.0 at 32, and 1,190.2 at 48 queued past the 32-slot cap. Throughput peaks at 32 users and drops when you queue beyond it; per-stream speed falls roughly halved in the queued 48-user run.

• AMD Strix Halo, DavidAU fusion Q6_K, llama.cpp with MTP 3: 15 to 22 tok/s generation, 162 tok/s prefill.
• RTX PRO 2000 Blackwell 16 GB, abliterated IQ3_S GGUF, llama.cpp with MTP: 87.8 tok/s aggregate across 4 slots,

600 tok/s prefill.

The pattern to take away: this is a dense model, so its speed tracks the memory bandwidth of the hardware. On datacenter-class cards it is genuinely fast, and its prefill is excellent; on bandwidth-limited desktop-class units it pays the full dense parameter cost on every token and lands at the back of the tier.

Quality and accuracy

Quality is where this model earns its place, and where the closed-book caveat matters most.

With retrieval, it ties the best model we test. On our CMMC RAG v3 battery, the FP8 build on the single FP8 build scored 0.971 strict in agentic mode, averaged over 3 seeds, statistically tied with GLM-5.3-Flash. The same records show 0.925 in code-first mode and 0.875 in frozen-passage mode, both over 3 seeds. A separate reference run from the v3 builder on the four-H200 box with TP4 scored 90 percent agentic strict, consistent with the ladder. In the same battery, gpt-oss-120b fell well behind and the 4 to 8B models fell well behind in agentic mode, so the 27B dense is in a different league once the model is allowed to drive retrieval itself.

Closed-book, it is weak, and we will not pretend otherwise. On the vendor dgx1 battery it answered 5 of 35 closed-book CMMC items correctly. The GB10 records note closed-book recall effectively absent at 1 to 4 of 35. The same pattern shows in the RAG ladder: 0.258 closed-book strict on one seed. If a workload depends on the model knowing compliance content from memory alone, this is the wrong model. With your own documents retrieved into context, that weakness mostly stops mattering, which is exactly how we run it.

Tool calling and long context are its best traits. On the dgx1 battery in BF16 it passed all 38 tool-call items, all 15 agent items, and all 26 long-context items, with verified context of the verified context depth confirmed by three needle passes. HumanEval came to 94.5 percent. On the single GB10 it was likewise the best tool, agent and long-context model of the one-unit tier, with 26 of 26 long-context items. On the Strix Halo sweep it was the highest scorer of the entire sweep, 38 of 40 MMLU-Pro and 20 of 20 HumanEval, just slow.

Hallucination in tool calls is not zero. The dgx1 battery recorded a nonzero rate of hallucinated calls on the multi-turn tool corpus in BF16. That matches several peers on the same battery, but it is a reason to keep tool schemas tight and to prefer the code-first wiring for high-stakes lookups, where fabrications drop to near zero for every model tested.

How we ran it

Every configuration in the tables was measured on our own fleet, from stored result files that name the exact engine build and flags. The GB300 runs used vLLM, one GPU, with BF16, FP8 and NVFP4 checkpoints measured on 2026-09-23 and 2026-09-25, including a concurrency ladder up to the engine sequence limit and a needle test at the verified context depth. The single GB10 run used vLLM nightly with MTP3 speculative decoding, the same speedgate battery as every other model on the one-unit tier: a gate first, then closed-loop decode at 1, 4, 8, 16 and 32 users with 512-token outputs at temperature zero, prose and code separately, and a ceiling defined as the most users who all hold at least 10 tok/s each. Quality batteries ran at vendor sampling. The H200 runs used the live FP8 service with TP=1, max_num_seqs 32 and MTP speculative decoding, re-measured on 2026-09-23 with the 09-21 run reproduced within noise. The Strix Halo run used unsloth llama.cpp with the MTP draft head under the Gufo toolbox on the 2026-10-13 sweep. Where a result was superseded, for example the SEO-shape runs at two sequence settings on the H200, we report the newer ladder and label the long-prompt runs as not comparable to the small-prompt one.

When to pick it

Pick Qwen3.8-27B when three things are true at once.

1. Your workload is grounded: the model reads your documents, your codebase or your compliance corpus through retrieval, and the quality bar is high. This is the model that matched the much larger flagship in agentic retrieval while fitting comfortably on one datacenter GPU in FP8. 2. You need long context that actually works. Verified needle retrieval at the record's verified context depth on the GB300 and the best long-context score on one GB10 unit make it a safe choice for whole-document analysis, contract review and log triage. 3. You can accept the closed-book limits. Anything that requires the model to know regulation content from memory will disappoint at 5 of 35, and on a single Spark unit you trade about three quarters of the speed of a Flash-class MoE model for quality.

Do not pick it as the interactive chat model on a single GB10; pick it as the batch, agent and long-context model there. Do not pick it for closed-book compliance Q&A anywhere; wire in retrieval first. On a GB300 or a single H200-class card, it is an easy default.

Where to go next

• LLM benchmarks hub
• Leaderboards
• How we measure
• Model tier list
• Private AI solutions
• Private LLM guide
• CMMC compliance

FAQ