Gemma-4-31B-ITbenchmark results

Measured speed and quality

Speed (single user)

Source: fleet-bench/2026-09-23-dgx1-dense-gemma4-31b-bf16.json, fleet-bench/2026-09-23-dgx1-dense-gemma4-31b-nvfp4.json, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-09-23-gb10-tp1-spark1-single-unit-survey.json, ai11/RESULTS-2026-10-13-model-sweep.md, fleet-bench/2026-09-27-strix-vs-gb10/README.md

Aggregate throughput

Source: fleet-bench/2026-09-23-dgx1-dense-gemma4-31b-bf16.json, fleet-bench/2026-09-23-dgx1-dense-gemma4-31b-nvfp4.json, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-09-23-gb10-tp1-spark1-single-unit-survey.json

Prefill and latency

Source: fleet-bench/2026-10-02-best-model-per-tier-results.md

Context

Source: fleet-bench/2026-10-02-best-model-per-tier-results.md

Quality

Source: fleet-bench/2026-10-02-best-model-per-tier-results.md

Power and efficiency

Source: fleet-bench/2026-10-02-best-model-per-tier-results.md

Other measurements

Source: ai11/RESULTS-2026-10-13-model-sweep.md, fleet-bench/2026-09-27-strix-vs-gb10/README.md

Gemma 4 is Google's open-weight model family, and on our fleet it comes in two shapes we actually measured: the 26B-A4B mixture-of-experts variant and the 31B dense instruction-tuned model. The two behave like different products. The 26B-A4B is the best quality among the mixture-of-experts models we ran on a single DGX Spark class unit, holding 64 concurrent users above a ten token per second floor while answering almost every tool and long document question correctly. The 31B dense model is the opposite trade: it scores beautifully on knowledge tests but reads its entire weight set on every token, which makes it punishingly slow on memory-bandwidth-limited hardware.

Petronella Technology Group, Inc. measured Gemma 4 on three hardware families: single and paired NVIDIA GB10 units (DGX Spark class), the DGX Station GB300, and AMD Strix Halo machines (the Minisforum MS-S1 MAX and an HP ZBook Ultra). Every number on this page comes from those runs; where a result failed a quality gate or looped at the sampling settings we used, we say so rather than quoting the speed as if it were usable.

What it is

Gemma 4 ships in at least two architectures that matter for private deployment. Gemma-4-26B-A4B is a sparse mixture-of-experts model: roughly 26 billion total parameters, but only about 4 billion active per token, so each generated token reads a small slice of the weights. Gemma-4-31B-it (also labeled Gemma 4 31B dense in our sweep) is a conventional dense model: every token requires reading the whole weight set. That one architectural difference explains nearly every speed number on this page.

The practical consequences we measured:

• On one GB10 unit running vLLM nightly with the NVFP4 checkpoint, the 26B-A4B decoded 31.5 tok/s for one user on prose and 31.4 tok/s on code, with no published draft model to accelerate it. That is the single biggest knock against it on Spark: it is roughly a third of the speed of the fastest mixture-of-experts models we tested there.
• The same 26B-A4B checkpoint scaled beautifully under load. Aggregate decode reached 552.8 tok/s at 32 users and 866.4 tok/s at its ceiling of 64 users, the most concurrent users of any model in our one-unit tier at the ten tok/s floor. It also posted the best energy efficiency of that tier at 6.63 tok/J.
• The 31B dense model on one GB10 in bf16 is a case study in what not to do. It fit (roughly 63 GiB consumed) and batched cleanly to a peak aggregate of 101.4 tok/s at 32 concurrent streams, but each individual stream crawled at 3.8 tok/s. Our survey note is blunt: bf16 on one GB10 is a waste of the box, because a GB10 is capacity-rich and bandwidth-poor and a dense model reads every weight byte per token.

Quality is where the 26B-A4B earns its place. In the 2026-10-02 one-unit tier battery it scored 37 of 38 tool calls correct, 14 of 35 on the CMMC control identification set, and 24 of 26 on long document questions, the best mixture-of-experts result of that round. In the grounded compliance battery it answered 59 to 62 of the strict set across three seeds with only 2 to 4 fabrications, and produced every tool call but one as a valid call across the whole battery. On the DGX Station GB300 the 31B dense model in NVFP4 hit 110 tok/s single stream and 3,076.2 tok/s aggregate at 32 users, which shows how differently the same family behaves when memory bandwidth is not the constraint.

Where it runs best

The honest answer: Gemma 4 wants hardware with bandwidth, and it wants to serve many users at once on Spark-class boxes.

• DGX Station GB300 (dgx1): the best measured home for Gemma 4. The 31B dense model in NVFP4 ran 110 tok/s for a single user and 3,076.2 tok/s in aggregate at 32 users; in bf16 the same model still managed 73.7 tok/s single stream and 2,111.0 tok/s at 32. If you own a workstation-class GB300 and want Gemma quality, this is where it shines.
• One GB10 unit with the 26B-A4B NVFP4 checkpoint: a strong multi-user server and a mediocre single-user assistant. Sixty-four users at the floor and 866.4 tok/s at the ceiling make it the best-sharing one-unit model we measured; 31.5 tok/s per person makes it a patient single user's model. The 128k prefill at 784 tok/s is slow, just under three minutes to first token on a very long document, so keep prompts short or accept the wait.
• One GB10 unit with the 31B dense bf16 checkpoint: do not do this. 3.8 tok/s per stream is unusable interactively, and even a speculative decoding rescue was not measured (no drafter was configured in our survey); the model merely fits, it does not serve.
• AMD Strix Halo (ai11, Minisforum MS-S1 MAX; g1a, HP ZBook Ultra G1a): mixed. The 26B MoE ran 51.8 tok/s median generation on g1a through Ollama with a long-prompt prefill of 1,190 tok/s, which is genuinely pleasant on a laptop. But quality there was a trap: at our greedy temperature-zero settings both the Q8 and the QAT Q4 checkpoints of the 26B looped, with 28 and 32 of 60 answers hitting the 16k output cap respectively, and MMLU-Pro scores of 14 and 10 of 40. The 31B dense Q8 on ai11 did not loop (only 3 answers at the cap) and scored 35 of 40 on MMLU-Pro and 19 of 20 on HumanEval, but at 18 tok/s with 130 tok/s prefill it is a patient-jobs model. On g1a the dense 31B was even slower at 8.8 tok/s.
• Windows on Strix Halo ran 20 to 30 percent slower than Linux in our sweep, so prefer the Linux side if you deploy on these boxes.

Measured speed

All numbers below were measured on our own fleet; the tables on this page carry the source files.

Single-user decode (one request, streaming):

| Configuration | Quant | Engine | tok/s | |---|---|---|---| | DGX Station GB300, 1 GPU, 31B dense | NVFP4 | vLLM | 110.0 | | DGX Station GB300, 1 GPU, 31B dense | BF16 | vLLM | 73.7 | | g1a Strix Halo laptop, 26B MoE | Q8_0 | Ollama (ROCm) | 51.8 | | 1x GB10 (DGX Spark class), 26B-A4B | NVFP4 | vLLM nightly | 31.5 | | ai11 Strix Halo, 31B dense | Q8 | llama.cpp Vulkan, MTP 3 | 18 | | 1x GB10 (DGX Spark class), 31B dense | BF16 | vLLM | 3.8 | | g1a Strix Halo laptop, 31B dense | default | Ollama (ROCm) | 8.8 |

The 26B-A4B QAT Q4 checkpoint with MTP 3 on ai11 was the fastest decoder in that whole sweep at 108 to 118 tok/s, but we do not count it as a usable result: 32 of 60 answers hit the 16k token cap in greedy loops, and MMLU-Pro fell to 10 of 40. Speed without correctness is not a benchmark result; a vendor-recommended-sampling rerun was queued and its numbers are not on this page.

Throughput under load, one GB10 unit, 26B-A4B NVFP4: 552.8 tok/s aggregate at 32 users, 866.4 tok/s at 64 users, and the same server still returned 31.4 tok/s to a single user while carrying that load. On dgx1 the 31B NVFP4 model reached 3,076.2 tok/s at 32 users with zero failed requests.

Prefill matters for long documents. On one GB10 unit the 26B-A4B prefilled 8k prompts at 6,247 tok/s, 32k at 2,689 tok/s, but 128k prompts dropped to 784 tok/s. That is the trade: short prompts feel fine, a 128k document costs minutes before the first token. A needle test at 243,271 tokens passed, and the same model still decoded at over twenty tok/s on a near-quarter-million-token prompt, so long context works; it is just slow to enter.

Quality and accuracy

The 26B-A4B on one GB10 unit was the best quality mixture-of-experts model of the 2026-10-02 round:

• Tools: 37 of 38 calls correct, and in the grounded compliance battery it avoided 9 or 10 of the trap questions across seeds.
• CMMC control identification: 14 of 35 strict, consistent across seeds at 13, 13 and 12. Not a compliance oracle, but far ahead of the other MoE models of that round.
• Long documents: 24 of 26, using a median of only 46 reasoning tokens per answer, the most token-efficient long-document behavior we measured.
• Grounded compliance battery: 59 to 62 of the strict set across three seeds, 2 to 4 fabrications, 9 or 10 trap questions avoided, work scored 8 to 9 of 10.
• Battery coding: 9 of 10, and writing 2 of 3.
• One caveat from the same run: needle answers sometimes carried a degenerate tail after the correct value. The gate passed on content, but check stop-sequence handling before serving it to users.

The dense 31B is the quality champion of the family on Strix Halo: 35 of 40 MMLU-Pro and 19 of 20 HumanEval on ai11 with almost no truncation. The MoE 26B failed our greedy sampling on that box (14 of 40 at best), which we attribute to looping under our sampling settings rather than to the model itself; treat the ai11 MoE quality numbers as provisional.

How we ran it

• GB10 (DGX Spark class): single unit on spark1 and msi1, vLLM nightly with the NVFP4 checkpoint for the 26B-A4B, and a development build of vLLM for the 31B bf16 survey. The 26B has no published draft model, so it ran without speculative decoding. Measured 2026-10-02 for the tier battery (with a September single-unit survey behind it), with a repeat run on a second GB10 build agreeing to within a third of a percent.
• DGX Station GB300: vLLM on one GPU, same-architecture bf16 versus NVFP4 pairs measured 2026-09-23 from the nvidia modelopt checkpoints.
• Strix Halo: unsloth llama.cpp Vulkan builds under Gufo with MTP draft 3 for the 31B dense and QAT Q4 26B, and Ollama with the ROCm backend on the g1a Windows laptop, measured 2026-10-13 and 2026-09-27 respectively. Quality batteries ran at temperature 0 with a 16k output cap; the 26B MoE looped under those settings on ai11, which is why we flag that configuration as failed rather than re-quoting its speed as a recommendation.
• Context: the 26B-A4B verified 243,271 tokens of context on one GB10 unit (its native limit is 262k tokens, and the KV pool held about five million tokens of headroom of headroom); about nineteen full-length requests ran at once and 64 users stayed above the floor.

When to pick it

Pick Gemma 4 26B-A4B on Spark-class hardware when correctness on tools, long documents and compliance-style grounding matters more than a snappy first token, and when several people share one box: no other one-unit MoE model we measured held 64 users at the floor with 6.63 tok/J of efficiency and that quality. It is the "quality at capacity" choice of the single-unit tier, not the "fast assistant" choice; if speed for one user is the goal, a model with a published draft model beats it by roughly three times.

Pick Gemma 4 31B dense on a DGX Station GB300, where NVFP4 turns it into a 110 tok/s single-user and 3,076.2 tok/s aggregate server. On bandwidth-limited boxes (one GB10 unit, or Strix Halo laptops), the dense 31B is a patient batch tool at best; its measured 3.8 tok/s on bf16 Spark and 8.8 to 18 tok/s on Strix Halo disqualify it from interactive use there.

Skip it entirely if you need cheap speed on one Spark unit: the same tier's draft-equipped models decode around three times faster, and the 26B Gemma's 128k prefill of 784 tok/s makes it the wrong tool for very large document ingestion on that hardware.

Where to go next

• LLM benchmarks hub
• Leaderboards
• How we measure
• Model tier list
• Private AI solutions
• Private LLM guide
• CMMC compliance

FAQ