Qwen3.6-35B-A3Bbenchmark results

Measured speed and quality

Speed (single user)

Source: fleet-bench/dgx1-overnight-2026-10-05/RESULTS.md, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-09-23-gb10-tp1-spark1-single-unit-survey.json, fleet-bench/2026-09-27-strix-vs-gb10/README.md

Aggregate throughput

Source: fleet-bench/dgx1-overnight-2026-10-05/RESULTS.md, fleet-bench/2026-10-02-best-model-per-tier-results.md, fleet-bench/2026-09-23-gb10-tp1-spark1-single-unit-survey.json

Prefill and latency

Source: fleet-bench/dgx1-overnight-2026-10-05/RESULTS.md, fleet-bench/2026-10-02-best-model-per-tier-results.md

Power and efficiency

Source: fleet-bench/dgx1-overnight-2026-10-05/RESULTS.md, fleet-bench/2026-10-02-best-model-per-tier-results.md

Other measurements

Source: fleet-bench/2026-10-03-cmmc-rag-v3-results.md, ai11/RESULTS-2026-10-13-model-sweep.md, fleet-bench/2026-09-27-strix-vs-gb10/README.md

# Qwen3.6-35B-A3B benchmarks on our fleet

What it is

Qwen3.6-35B-A3B is a sparse mixture-of-experts model: roughly 35B parameters total with only a few billion active per token, which is why it decodes far faster than a dense model of similar total size. Petronella Technology Group, Inc. measured it on three different hardware families in our own lab: a single GB10 unit (DGX Spark class), the DGX Station GB300, and the AMD Strix Halo class machines (ai11 and the g1a laptop). It is a tool-heavy generalist: in our dgx1 quality battery it answered all but one of the tool-calling items, the best tool result of any MoE model we tested alongside Gemma-4-26B-A4B, and it completed agentic tasks at a high rate with low fabrication.

The honest headline: it is one of the fastest single-user models we have measured on GB10 hardware, but it has a verified closed-book compliance-recall weakness and a runaway-reasoning failure mode on long documents. If you serve it as-is with a fixed output budget, those two weaknesses decide where it belongs, and where it does not.

Where it runs best

It runs best on a single GB10 unit with vLLM nightly and the NVFP4 quantization from NVIDIA. In that configuration it reaches 88.2 tok/s prose and 115.7 tok/s code for one user, and it sustains 452.9 tok/s aggregate at 32 users with a ceiling of 48 users. That combination of single-user speed and a real multi-user ceiling makes a one-unit GB10 deployment a strong shared assistant box for a small team, as long as the work is tools, code and grounded retrieval rather than closed-book compliance recall. Concretely, that is an internal coding assistant, an agent runner that calls external tools, or a retrieval-backed question answering service. The prefill numbers back this up: 7,492 tok/s on 8k prompts means a typical tool-result refill before the next agent step costs well under a second, which keeps agent loops feeling immediate.

The same model also runs well on Strix Halo machines. On ai11 (the large-memory Ryzen AI Max+ desktop, running llama.cpp Vulkan with MTP 3 and the UD-Q8_K_XL build) it generated in the 64 to 72 tok/s range, and on the g1a laptop with Ollama it reached 54.4 tok/s. Those speeds are comfortable for one person working with an assistant all day, and the big-memory Strix Halo machines hold the model with room to spare for long contexts. What they do not offer is batching: treat a Strix Halo deployment as one to a few seats, not a team server. On the DGX Station GB300 it is dramatically faster still: 335 tok/s single-user and 10,653 tok/s peak aggregate in BF16, though few buyers of this model own that class of hardware.

Choose the configuration by seat count and budget. A one-unit GB10 at the NVFP4 recipe is the value pick for a team of up to a few dozen concurrent users. A Strix Halo desktop is the pick when the machine must also be quiet, office-friendly and general purpose, or when you want a fully offline workstation. The GB300 is only worth naming if you already own one; at 895 W under load it is a rack decision, not a desk decision.

Measured speed

Measured on 1x GB10 (DGX Spark class), vLLM nightly, NVFP4: one user decodes prose at 88.2 tok/s and code at 115.7 tok/s. At 32 users the aggregate decode is 452.9 tok/s, and the ceiling is 48 users at 507.6 tok/s aggregate. Prefill is 7,492 tok/s on 8k prompts, 6,021 tok/s on 32k prompts and 3,117 tok/s on 128k prompts. Energy efficiency is 3.62 tok/J. A needle test at nine tenths of the served maximum context passed, and at a near-maximum prompt the model still decoded at a healthy rate for one user under speculative decoding.

The older single-unit survey on the same hardware class with Ollama and a Q4_K_M GGUF measured 42.2 tok/s single-stream and a peak aggregate of only 57.9 tok/s at 8 users. The reason is not the model, it is the engine: Ollama refuses to batch this architecture, so the Q4_K_M Ollama build is a single-seat option only. Do not put it behind a queue. The vLLM NVFP4 recipe is what turns this model into a multi-user service.

Nemotron-3.5-Lightning was marginally faster on one GB10 unit for a single user, while scoring worse on tools and quality, so Qwen3.6-35B-A3B is roughly the fastest one-unit model we measured that still carries a strong tool profile.

Measured on 1x GB300 (DGX Station GB300, dgx1), vLLM, BF16: 335 tok/s single user, 4,359 tok/s aggregate at 32 users, 10,653 tok/s peak aggregate, prefill of 39,068 / 80,337 / 65,231 tok/s at the 8k / 32k / 128k prompt sizes, 895 W at the wall and 4.87 tok/J. This is the configuration for anyone who needs dozens of concurrent assistants without quantization worries, and the BF16 run passed every correctness gate we ran.

Measured on Strix Halo (AMD Ryzen AI Max+ class):

• ai11 (the Minisforum desktop built around the large unified-memory Ryzen AI Max+ PRO chip), llama.cpp Vulkan with MTP 3, UD-Q8_K_XL: generation between 64 and 72 tok/s, prefill around 1.07k tok/s. In our 60-item quality battery it scored 35 of 40 on MMLU-Pro and a perfect 20 of 20 on HumanEval. It was the fastest good model in that sweep.
• g1a (an HP ZBook Ultra laptop with the smaller Ryzen AI Max+ PRO chip and Radeon 8060S graphics), Ollama with ROCm, Q4_K_M: 54.4 tok/s median generation and 980 tok/s on the long-prefill prompt.
• g1a with llama.cpp Vulkan, Q4_K_M: prefill of 1,313 tok/s and 59.4 tok/s tg128 at depth zero, falling to 1,028 and 56.4 at an 8k context and 638 and 49.4 at 32k. With ROCm instead of Vulkan the same battery measured 1,200 and 54.3 at depth zero and 760 and 44.7 at 32k, so Vulkan is the faster backend for this model on g1a.

Quality and accuracy

This is where the page must be blunt. In the dgx1 battery the model scored at the top of the MoE field on tools (only one miss out of 38) and dropped just one code item, and its agentic completion rate matched the other top models with low fabrication. But closed-book CMMC control recall was 1 of 35 on the scored run, with a mean in the low single digits across repeats: a real recall weakness. Our verification of the raw outputs showed every answer was a well-formed control ID after thousands of tokens of reasoning; the model simply picked the wrong control (for example, answering with a neighboring control's ID). Do not use it as a closed-book compliance assistant.

Long-document work as served also fails: in the long-context suite, all but one of its failures returned an empty answer because the model was still reasoning when the fixed output budget ran out on long-document questions, and the last failure was a degenerate repetition loop, also at the budget. The needle test passes because a needle answer needs little reasoning. This is output-budget exhaustion by runaway reasoning, not a parser fault. If you deploy it for long-document Q&A, either raise the output budget well past the suite's limit, disable or bound the reasoning phase, or pick a different model; whether it answers correctly with a larger budget was not measured by us.

Grounded compliance work is different. In the CMMC RAG v3 agentic-strict evaluation on one GB10 unit it scored 0.735, ahead of Gemma-4-26B-A4B and Nemotron-3.5, behind gpt-oss-120b and well behind the dense Qwen3.8-27B, and its answer-grounding rate was the best in that tier row. With retrieval feeding it the right control text, the recall weakness mostly disappears. The pattern for buyers: fine with a retrieval layer in front of it, unsafe without one.

On ai11 it scored 35 of 40 on MMLU-Pro and a perfect 20 of 20 on HumanEval in the same battery, which fits the profile: strong reasoning and code, uneven recall from memory.

How we ran it

Every number on this page comes from our own fleet, never from model cards or vendor claims.

• GB10 (DGX Spark class), one unit: vLLM nightly with the NVIDIA NVFP4 checkpoint under our RedHat DGX Spark Kubernetes recipe. The run passed all correctness gates. We also cross-checked a second GB10 build from a different vendor at the same locked clock and found decode within about one percent, so the numbers are not specific to one unit.
• Ollama survey: Ollama, Q4_K_M GGUF, measured as part of a single-unit survey across models.
• DGX Station GB300 (dgx1): vLLM, BF16, measured overnight with every correctness gate passing; power measured at the wall.
• Strix Halo ai11: llama.cpp Vulkan with multi-token prediction enabled, unsigned-dequant UD-Q8_K_XL quantization, scored with a 60-item MMLU-Pro plus HumanEval battery.
• Strix Halo g1a: both Ollama with ROCm and llama.cpp under Vulkan and ROCm, Q4_K_M, with the same files and build as the NVIDIA-side comparison of that week.
• CMMC RAG v3: the agentic-strict variant, one GB10 unit, single seed.

Methodology details, prompt shapes and gate definitions are on the methodology page linked below.

When to pick it

Pick Qwen3.6-35B-A3B when you want nearly the fastest single-user model on a one-unit GB10 box with genuinely excellent tool calling, and you will put a retrieval layer in front of it. It is our pick for a small-team shared assistant that spends its day on tool calls, code generation and grounded document Q&A.

Do not pick it when the workload is closed-book compliance recall (it scored as low as 1 of 35) or unattended long-document answers under a fixed output budget (nearly every long-document failure was reasoning that outlived the budget). If you need a compliance-first or long-document model on the same hardware, look at Gemma-4-26B-A4B for quality or the dense Qwen3.8-27B for the best tool, agent and long-context scores at a quarter of the speed.

Avoid the Ollama Q4_K_M build for anything shared: it cannot batch this architecture, so peak aggregate was 57.9 tok/s at 8 users. One seat, or switch to vLLM.

On Strix Halo it is the fastest good model we measured on ai11 and a strong choice for a workstation or laptop assistant; prefer the Vulkan backend on g1a, and prefer multi-token prediction if your build supports it.

A practical rollout plan for the common case, a one-unit GB10 team server with retrieval: serve the NVFP4 checkpoint with vLLM, cap concurrent sequences near the measured ceiling rather than at it so late users do not queue behind a full engine, set the served context to what your retrieval chunks actually need, and give long-generation requests an explicit, generous output budget so the runaway-reasoning failure mode surfaces as a slow answer instead of an empty one. Wire every compliance-flavored question through the retrieval layer, never let the model answer control questions from memory, and log the retrieved source with each answer so you can audit what the model actually read. With that framing this model is a dependable daily driver; without it, the same weights will quietly produce wrong compliance citations at full speed.

If your workload is mostly code, the code decode rate of 115.7 tok/s against 88.2 tok/s prose matters more than any quality battery: code comes out of this model noticeably faster than prose because the speculative decoding accepts more aggressively on structured text. Pair it with your editor's inline completion and a terminal agent and the model rarely feels slower than a hosted service.

Where to go next

• LLM benchmarks hub
• Leaderboards
• How we measure
• Model tier list
• Private AI solutions
• Private LLM guide
• CMMC compliance

FAQ