Hardware Deep Dive · Measured September to November 2026

GB10 Cluster Benchmarks:One, Two, Four and Eight Units, Measured

Short answerOne DGX Spark class box (128 GB of unified memory at 273 GB/s) is a machine for one to eight people, and the model you pick matters more than how many boxes you buy. Nemotron 3.5 Bolt on the TensorFold engine answered one user at 72 tokens per second and carried 16 users at 258 tokens per second, while a dense 27B on the same box managed 15 tokens per second without its drafter and 49 with it. More boxes buy model size and concurrent users rather than single-user speed: GLM 5.3 Flash on our vLLM build went from 25 tok/s on four boxes to 33 on eight (1.32x), while the newer community TensorFold recipe reached 77.2 tok/s on four, and DeepSeek V4.1 Flash has no recipe for fewer than four. Buy one for a desk, four linked together for a team that needs a large model plus real prompt-reading speed, and something else entirely if your workload is one person and a Mac fits the model.

Most reviews of these appliances time a single box on toy prompts, which tells a buyer nothing about what happens when you add a second, third or fourth box. We operate eight of them on a switched 200 Gb fabric, so we could ask the question buyers actually have: at what point does buying another box stop paying? Every number on this page comes from a stored result file on hardware we own, measured with our published benchmark methodology.

8 units measured|1 / 2 / 4 / 8-unit topologies|Correctness gate before every row|128k windows
72.4 tok/s
Nemotron 3.5 Bolt, one user, one unit (TensorFold)
10,052
prefill tok/s, DeepSeek V4.1 Flash on four units (Pico)
1.32x
single-user gain, four units to eight (GLM 5.3 Flash)
538 tok/s
aggregate at 32 users, one unit (gpt-oss-20b, vLLM)
NVIDIA DGX Spark class desktop AI appliances clustered for local LLM serving, measured by Petronella Technology Group, Inc.
The short answer

What Each Cluster Size Is For

One row per topology. Single-user speed is steady decode for one request. Aggregate is the total tokens per second at the busiest level we measured for that configuration. Every row passed an arithmetic and long-context needle gate before it was timed, and every number traces to a result file named in the Sources section.

TopologyBest model we measuredEngineOne userAggregate at loadWhat it is for
1x GB10 (DGX Spark class)Nemotron 3.5 Bolt (Mia recipe)TensorFold72.4 tok/s258 tok/s at 16 usersA desk: one person or a small team, 128 GB of context room
1x GB10Qwen 3.8 27B + DFlash2TensorFold 1.2.748.7 tok/s182 tok/s at 8 usersBest dense-model quality on one box
1x GB10gpt-oss-20bvLLM45.8 tok/s381 tok/s at 16 usersFastest prompt reading on one unit (5.4 s for 32k)
2x GB10 (TP2)Qwen3.8 Flash NextvLLM, MTP337.5 / 53.1 tok/s (prose / code)131 prose at 8 usersA pair for one to four users on a bigger MoE
4x GB10 (TP4)DeepSeek V4.1 Flash (Pico)Pico pico-dgx-ds480.8 tok/s232 tok/s at 8 users; prefill 10,052 tok/sA team on a large MoE with agent-grade prefill
4x GB10 (TP4)GLM 5.3 FlashvLLM, DFlash225.3 tok/s221 tok/s at 32 usersQuality parity with our data-center GPUs on a big MoE
8x GB10 (TP8)Qwen3.8 Flash NextvLLM, MTP4 + EP47.5 tok/s538 tok/s at 32 users64-user serving on one engine; GLM scales better per added unit

The 2, 4 and 8-unit rows come from our September concurrency-knee runs; the four-unit Pico and TensorFold rows come from the November trial and production lanes. The engines and quantizations differ by row because that is what each topology actually serves; we label the configuration on every row instead of projecting a like-for-like number we do not have.

Sizing a private deployment? See how we build on-premises AI
The hardware

What a GB10 Is and Why GB10 Cluster Benchmarks Differ by Unit Count

The NVIDIA DGX Spark and the MSI EdgeXpert MS-C931 are the same class of machine: both are built on NVIDIA's GB10 superchip with 128 GB of LPDDR5x unified memory on a 256-bit interface at 273 GB/s of bandwidth (NVIDIA's own hardware specification, linked in Sources). There is no separate video memory; the CPU and GPU share one pool, which is what lets a desk-side box hold a model that would need multiple discrete GPUs otherwise. The trade is bandwidth: 273 GB/s is roughly a third of a Mac Studio M5 Max's 614 GB/s and a fraction of a data-center GPU's 4 to 4.8 TB/s, so single-user decode on dense models is modest no matter what you do in software.

Eight of these boxes sit in our rack, evenly split between the NVIDIA and MSI builds, on a 200 Gb RoCE switched fabric. That fleet is the whole reason this page exists: most reviews test one unit with toy prompts. We have run every topology from one unit to eight, under concurrency ladders, with a correctness gate before every timed row.

Two facts from our fleet data shape everything else on this page. First, of the models people actually want, almost nothing big fits one unit: the NVFP4 build of DeepSeek V4.1 Flash is 492 GB (4.1x one unit), Qwen3.8 Flash Next NVFP4 is 126 GB (does not fit before the KV cache even starts), and GLM 5.3 Flash NVFP4 is 185 GB. Second, what fits is often not usable: a 72 billion parameter dense model at 4-bit loads on one unit and answers at 4.4 tokens per second, collapsing further under load, because a dense model of that size is bandwidth-starved on 273 GB/s. The shape that suits this hardware is a small-active mixture-of-experts model, or a dense model paired with a speculative decoder.

One unit

One DGX Spark: Best Models and Real Numbers

We surveyed everything staged on one unit in September, then re-ran the winners under the full methodology in November. The ladder below is the September single-unit survey (closed-loop decode, 256-token answers, temperature 0, a distinct prompt per request so no cache flatters the result).

Model on 1x GB101 user8 users (agg)16 users (agg)32 users (agg)Engine
Nemotron 3.5 Lightning 30B-A3B, NVFP472.1253369514vLLM nightly
Nemotron 3.5 Bolt (Mia TensorFold recipe)72.4see the hub page258not re-runTensorFold
Qwen 3.6 35B-A3B, Q4_K_M GGUF42.257.957.857.8 (queued)Ollama
Gemma 4 31B, Q4_K_M GGUF10.356.2107176Ollama
Gemma 4 31B, bf163.828.956.5101vLLM
Qwen 2.5 72B, Q4_K_M GGUF4.410.927.714.4 (collapse)Ollama

The three models worth running on one box

Nemotron 3.5 Bolt is the best all-rounder we measured. On the TensorFold engine with the community recipe, it passed every gate question, recalled a planted fact at 16k, 120k and 250k tokens, and read prompts above 2,000 tok/s from 4k to 120k (2,436 at 32k). One user saw 72.4 tok/s; sixteen users together pulled 258 while each still got about 17. The honest limit: when sixteen agents each dropped a fresh 32k-token request at once, all sixteen completed, but the last one waited about 219 seconds to start. One box reads sequentially under a burst, and that is what more units are for.

Qwen 3.8 27B with the DFlash2 drafter is the dense pick. Without the drafter it is 15.1 tok/s for one user; with it, 48.7, a 3.2x speedup that persists under load (182 aggregate at 8 users against 89 without). Prefill does not change with a drafter, so a 32k prompt waits about 26 seconds either way. On our quality battery it missed one of eight gate questions (a five-digit multiplication) and passed both long-context needles, and its CMMC retrieval accuracy of 0.978 was the best single-box score we recorded.

gpt-oss-20b is the reading and concurrency specialist. It started answering a 32k prompt in 5.4 seconds and served 381 aggregate tok/s at sixteen users (538 at 32). Its quality gate was being re-run at this writing because the first pass used an output budget that cut its reasoning short, so treat its quality score as pending; its speed rows stand.

What does not work on one unit

Dense large models are the trap. The 72B at 4-bit fits in memory and is unusable (4.4 tok/s, collapsing under concurrency). Gemma 4 31B in bf16 runs but at 3.8 tok/s per stream. The mixture-of-experts Qwen 3.6 35B-A3B at 4-bit holds about 42 tok/s for one user but its Ollama serving profile queues everything past a few users. And the headline pod models (GLM 5.3 Flash at 185 GB, DeepSeek V4.1 Flash at 492 GB, Qwen3.8 Flash Next NVFP4 at 126 GB) simply do not fit. A smaller MoE at NVFP4, or a dense 27B with a drafter, is what one GB10 is for.

Two units

Two Sparks: the Pair That Mostly Makes Sense for One Model

Tensor-parallel two is the least predictable tier, because an 185 GB checkpoint loads 89.2 GiB per node and leaves almost nothing for context. The pair earns its keep only when the model's per-rank weight load leaves real KV room.

Model, 2x GB101 user4 users8 users16 usersVerdict
Qwen3.8 Flash Next, NVFP4, MTP3 (SPEED profile)33.198.9140.8201.3Best pair model; profile caps at 6 sequences by design
MiMo-V2.6-Flash, NVFP424.367.898.5137.4Works, slower than Flash Next
GLM 5.3 Flash, NVFP4 (185 GB checkpoint)does not fit a pair: 89.2 GiB of weights per node leaves 0.25 GiB of usable KV cachevLLM refuses to start

The Flash Next pair reproduced its published recipe speed on code-like text (53.1 tok/s in our later tier contest against the recipe's 53.7 headline) with the best Qwen quality score of any lane we ran that week, but the profile is a single-user design by its own author's intent: six sequences maximum, so from eight users upward requests queue and aggregate goes flat. It is a one-to-four-user configuration. The same pair, serving profile, holds about 64 users.

The honest buying advice for a pair: buy two units when the model you want loads about 40 to 60 GiB per rank at your target context, or when you want two single-box replicas instead of one split model. Two replicas of a one-unit model nearly always beat one split model for total throughput, because splitting adds network synchronization to every token. We measured GLM 5.3 Flash scaling from four to eight units at 1.43x aggregate (1.26x for Flash Next over the same step), and the same physics applies at the two-unit step: the split costs you something before it pays you back.

Four units

Four Sparks: the Cluster That Earns Its Footprint

Four units is where the large mixture-of-experts models land, and where the engines differ enough to matter. Same model, same weights, same four boxes, three engines.

DeepSeek V4.1 Flash on 4x GB101 user8 users (agg)Prefill 8k / 32k / 128kEngine
Pico pico-dgx-ds4 (production lane)80.8232.48,929 / 9,801 / 10,052Pico C++/CUDA engine
Same model, Mia's TensorFold recipe, FP865.0162.63,155 / 3,437 / 3,381TensorFold native
Same model, Mia's recipe, NVFP4 experts60.8151.12,876 / 3,186 / 3,227TensorFold native

Pico's engine is the prefill story of the year on this hardware. The same four-unit cluster that read prompts at about 3,400 tok/s under TensorFold read them at 8,900 to 10,000 tok/s under Pico, roughly three times faster, with single-user decode also ahead (80.8 against 65.0) and aggregate at eight users up 43 percent. It passed our text, thinking, tool-call and 113k-token needle checks. The one caveat we publish: its batch-aware expert re-route setting means replies under concurrency can differ slightly from the solo reply; drop that flag for strictly lossless multi-user output at some speed cost.

GLM 5.3 Flash on four units is the quality story. Its NVFP4 checkpoint (185 GB) does not fit a pair, so four is the minimum. On our full evaluation battery it matched our data-center rows: 26 of 35 closed-book CMMC (against 26 on the DGX Station GB300 and 28 on the four-way H200 pool), 88.9 percent tool-calling with not one hallucinated call, plus perfect scores on both the code and the long-context batteries. Speed at TP4 was 25.3 tok/s for one user and 221 aggregate at 32 users in vLLM with the DFlash2 drafter; the community TensorFold recipe later measured 77.2 single / 173 at eight streams on a four-unit ring. If your work is quality-sensitive compliance or code generation on a large MoE, this is the four-unit lane.

Qwen3.8 Flash Next at TP4 is the throughput story: 49.4 prose / 75.1 code for one user, 675 aggregate at 64 users, prefill 3.8 to 4.0k tok/s. It also solved 12 of 12 agentic coding tasks at a median 62 seconds each. But its closed-book compliance recall was the weakest of the three (11.8 of 35), so we route Flash Next to bulk work, not to compliance answers.

Eight units

Eight Sparks: Where Scaling Stops Paying

The eight-unit pod (four DGX Spark plus four MSI EdgeXpert, one rank each on the 200 Gb fabric) is the largest topology we run. It serves a 64-user production profile on Flash Next. The numbers say when to stop adding boxes.

Model, TP4 vs TP81 user (TP4 / TP8)32 users agg (TP4 / TP8)Speedup, TP8 over TP4
GLM 5.3 Flash, NVFP425.3 / 33.3220.7 / 316.71.43x aggregate, 1.32x single
Qwen3.8 Flash Next, NVFP444.4 / 47.5426.6 / 537.61.26x aggregate, 1.07x single

The pattern held across every model we scaled: doubling from four to eight units bought 26 to 43 percent aggregate and 7 to 32 percent single-user speed, never the 2x the unit count suggests. Splitting a model across a network costs synchronization on every token, and at TP8 each rank also holds only a fraction of the experts. The eight-unit pod's real value is headroom: it holds 2.5 million KV tokens at 32k context for GLM class models, so it serves 64 concurrent users at a 20 tok/s floor. If you are buying new, four well-fed units of the right model beat eight of the wrong one.

One more eight-unit finding we publish because it saved us a week: DeepSeek V4.1 Flash (the full 492 GB NVFP4 checkpoint) does not serve on GB10 at TP8 under vLLM nightly. The blocker is kernel availability in that build rather than memory, fabric or configuration; the same model serves fine on four units under Pico or TensorFold with its community rank-file format. Before you plan a large vLLM deployment on GB10, check the engine's kernel support for your exact checkpoint class, not just its parameter count.

Prompt reading

Prefill: the Number That Decides Agent Work

Decode speed is what a chat feels like; prefill is what an agent feels like, because an agent re-reads its context every turn. On the four-unit pod with Qwen3.8 Flash Next, time to first token measured 6.5 seconds at 16k tokens and 37.9 at 120k, holding about 3,100 to 3,500 tok/s across the whole curve with the prefix cache defeated. Under a burst of eight agents each sending a full 128k context, the first tokens still came at about 3,100 aggregate tok/s but the last agent waited 337 seconds: prefill is the queue, not the model.

That is also where unified-memory machines that are not GB10s fall behind. Our 192 GB Ryzen AI Max+ PRO 495 box reads prompts at roughly 180 tok/s with GLM 5.3 Flash resident, an order of magnitude under the four-unit Spark's 10,000. If your workload is long-context agent loops, prefill speed, not model capacity, is the spec to buy. The full curve is on the best local LLM hub page.

Engines and recipes

Engines on GB10: vLLM, SGLang, TensorFold, Pico and the Mia Recipes

The same model on the same boxes moves by a factor of three across engines. This table is the map of what we ran and what each is for; every recipe author is credited on the hub page's recipe-credits section.

EngineWhat it isWhere it won for usWatch out for
vLLM (nightly, aarch64)The standard serving engine; NVFP4 and MXFP4 checkpoints, tensor and expert parallel, the eight-unit ladderServing profiles to 64 users; gpt-oss and Qwen MoE checkpoints; the most configurableKernel support for specific checkpoint classes on sm_121 must be checked per model (DeepSeek V4.1 TP8 does not serve)
SGLangAlternative serving engine with its own drafter ecosystemCommunity MiMo recipes on the pod run itWe have no published SGLang head-to-head on one GB10; say so rather than infer
TensorFold (Ash Hart)Native engine with speculative drafters (DFlash2) and Apple Metal / AMD HIP buildsFastest dense-model path on one box (Qwen 27B + DFlash2, 3.2x) and the cross-platform recipesRecipe-pinned checkpoints; a recipe tied to one revision can break on the next update
Pico (pico-dgx-ds4)C++/CUDA engine with a Rust front end for DeepSeek V4.1 Flash on four SparksPrefill 3x TensorFold on our fabric; engine ready in 14 secondsBatch-aware expert re-route makes concurrent replies differ from solo; key handling differs from the standard stack
Mia's AI Lab recipesPublished TensorFold launch configurations for DeepSeek, GLM and Flash Next on 1 to 4 SparksThe reference point our four-unit DeepSeek lane ran; her published survey is the best public cross-recipe tablePublished headline numbers are code-like text; prose measured 30 to 50 percent lower on every lane we reproduced

Two recipe-level findings change buying decisions. First, speculative decoding is not a gimmick on this hardware; it is the difference between a dense model being unusable and pleasant, and it is free quality-wise (verified output identical, to the character, in our Mac runs of the same engine family). Second, drafter licenses are part of the bill of materials: the GLM DFlash2 drafter was non-commercial at the revision most recipes pinned until it was relicensed Apache 2.0 in November 2026, and community repacks still carry the old tag. Check the drafter revision, not just the model's license.

Honest verdicts

When Not to Buy a DGX Spark

Craig asked us to publish this plainly: if another machine we measured beats the Spark for your workload, say so.

Buy a Mac Studio instead when

One person wants the fastest single-box replies and no Linux admin.

The Mac Studio M5 Max 64 GB answered one user at 133 to 135 tok/s on Qwen 3.8 27B with the DFlash2 drafter, against 48.7 on one GB10 with the same drafter family, and it does it silently on a desk. It loses on parallel agents and on 128 GB context room per stream. Details on the hub page.

Buy Strix Halo instead when

The budget is tight and the box must also be a general-purpose PC.

A Ryzen AI Max+ 395 machine is the cheapest route to 128 GB of unified memory and runs Windows or Linux as a normal computer. On the same llama.cpp build and same model files, our GB10 generated 17 to 41 percent faster and read prompts 1.8 to 3.8x faster; if those are not your bottleneck, the cheaper box wins.

Buy an RTX card instead when

The models fit in VRAM and one user at a time is the job.

An RTX PRO 6000 (96 GB) ran Qwen 3.8 27B at 80.1 tok/s against 24.4 on one GB10, and gpt-oss-120b at 179 against 35, on the same models. It sits in an existing desktop. It does not cluster cheaply and it does not hold 128 GB of context per stream. The comparison is on our GB10 vs H200 page.

Skip dedicated hardware when

A fallback box is the actual requirement.

If production inference already runs elsewhere and the need is a resilience box for an outage, a unified-memory desktop usually covers it at a fraction of the spend. We tell clients this when the honest answer is that the second GPU rarely justifies itself. The decision is budget and risk, not benchmark speed.

And when the Spark is right: CUDA and the NVFP4 toolchain, prefill several times faster than any other unified-memory option at this size, and clusters that scale to large MoE models over 200 Gb links. That combination is why we run eight.

FAQ

DGX Spark Questions, Answered From Our Measurements

Is a DGX Spark good for running local LLMs?

Yes, for the right model shape. One unit runs small-active MoE models (Nemotron 3.5 Bolt at 72 tok/s for one user) and dense 27B models with a speculative drafter (Qwen 3.8 27B at 49) well, with 128 GB of context room. Large dense models are bandwidth-starved on it, and models above about 100 GB of weights need a cluster.

How many DGX Sparks do I need?

One for a desk, if a Mac would not fit your model. Two for a 40 to 60 GiB-per-rank MoE or two replicas. Four to run DeepSeek V4.1 Flash or GLM 5.3 Flash at production quality with fast prefill. Eight only buys 26 to 43 percent more aggregate over four, so it is headroom, not a doubling.

Does adding a second Spark make one user faster?

Not much. In our TP4 to TP8 scaling, single-user speed rose 7 to 32 percent when units doubled. Units buy model capacity and concurrent users. If one user is the workload, buy the fastest single box that fits the model instead.

DGX Spark vs Mac Studio for one user?

The Mac decodes faster for one user (133 to 135 tok/s on Qwen 3.8 27B with a drafter, against 49 on one Spark with the same drafter family) and runs silently on macOS. The Spark reads prompts several times faster and holds more context per stream in its 128 GB. One user, small prompts: Mac. Agents and long documents: Spark.

DGX Spark vs Strix Halo?

On the same llama.cpp build and same model files, our GB10 generated 17 to 41 percent faster and processed prompts 1.8 to 3.8x faster than the 128 GB Ryzen AI Max+ 395 laptop. Strix Halo is the cheaper box and a general-purpose PC; the Spark is the faster inference appliance with CUDA.

Can a DGX Spark run DeepSeek or GLM?

On four units, yes: DeepSeek V4.1 Flash at 80.8 tok/s single-user with about 10,000 tok/s of prefill on the Pico engine, and GLM 5.3 Flash with quality equal to our data-center GPUs (26 of 35 closed-book CMMC, zero hallucinated tool calls). Neither fits one or two units in the checkpoints we tested.

What is the best model on one DGX Spark?

For breadth, Nemotron 3.5 Bolt on the TensorFold recipe (72.4 tok/s one user, 258 aggregate at 16 users, recall to 250k). For dense quality and compliance retrieval, Qwen 3.8 27B with the DFlash2 drafter (0.978 CMMC RAG accuracy, the best single-box score we recorded). For prompt-reading speed, gpt-oss-20b.

How much power does a DGX Spark use?

Our rack power meter read 127 to 158 W per unit while generating and about 45 to 60 W idle, so a desk circuit handles even a four-unit cluster with room to spare. We do not publish tokens-per-watt figures; power was not measured under identical load on every machine we compare against.

Do I need the 200 Gb cluster fabric?

Only for tensor-parallel clusters. A second network port measured exactly 1.00x on decode for a two-unit pair, and single units never touch the fabric. For one- or two-box setups, ordinary Ethernet is enough; the switched fabric is for TP4 and TP8 engines.

About the author

Craig Petronella

Craig Petronella founded Petronella Technology Group, Inc. in 2002 and leads its private AI and compliance work. He is a CMMC Registered Practitioner (CMMC-RP), Cisco CCNA, Certified Wireless Network Expert (CWNE) and North Carolina Digital Forensic Examiner (License 604180-DFE), and the author of 16 books. His team operates the eight-unit GB10 fleet behind these measurements.

Keep comparing

More Hardware Deep Dives From Our Benchmark Lab

Every page below uses the same prompts, the same quality gate and the same methodology, so the numbers line up across machines.

Sources

Sources and References

Sizing a Spark, a Cluster, or Something Else?

We have run every topology on this page in production, and we can benchmark your model and your user counts against them before you spend. Private AI on hardware you control is what we build.

Prefer to read first? Explore on-premises AI server builds.

Measured on the Petronella Technology Group, Inc. fleet between September and November 2026. Every number traces to a stored result file whose engine version, checkpoint and launch flags were read back from the running server.