You have two NVIDIA DGX Spark units on the desk, one direct QSFP cable between them, and one model you want served by both of them. This post is the recipe we use to make those two units serve one model as a single engine. It is the two-unit configuration Petronella Technology Group, Inc. measured on its own DGX Spark pod on 2026-10-02, and as of 2026-10-26 the whole thing is open source.
The recipe serves RadixArk's NVFP4 quantization of Qwen3.8-Flash-Next across both units with tensor parallelism 2, expert parallelism, and 4-token multi-token prediction, with NCCL running over RDMA on the cable. Every command below comes from the public repository, which is MIT licensed. We explain what each setting does, why it is there, the failure you are most likely to hit, and the honest limits of the measurement. If you want the model running today and would rather not debug the transport yourself, call Penny at 919-348-4912 or use our contact form and our team will set it up with you.
Key takeaways
- Two DGX Spark units joined by one direct QSFP cable serve Qwen3.8-Flash-Next (NVFP4) as one tensor-parallel engine with the open recipe, and a correctness gate passed 8 of 8 before any speed was recorded.
- One user sees 35.2 tok/s on prose and 52.6 tok/s on code; 32 simultaneous users aggregate 329.6 tok/s, and 32 is the most served at our 10 tok/s per-user floor.
- Prefill is nearly flat across prompt sizes: 3,089 tok/s at an 8K-token prompt and 3,069 tok/s at 32K.
- The pair drew 251 W at 1 user and 285 W at the ceiling, 1.16 tokens per joule, which makes two units the most efficient layout per joule of the layouts we measured.
- Long-context needle retrieval passed at every tested length up to 120,001 tokens, so the served 131,072-token window is real, not a spec-sheet number.
What the open-source recipe gives you
The qwen3.8-flash-next-dual-dgx-spark repository is deliberately small. It contains launch/node.sh, a script that starts one rank of the engine and which you run on both units; scripts/smoke-test.sh, which proves the pair actually serves after startup; and a README that explains the same configuration this post walks through. There is no magic in the repo, and that is the point: you can read every line of what touches your hardware in an afternoon.
One thing the recipe is not is the container image. The image is oxbyte/qwen3.8-flash-next-dual-spark, and oxbyte packages vLLM for this model on dual Sparks. Our recipe adds the launch procedure, the transport settings, and the measurements; the image tag is latest, and we record the image ID we measured so you can compare your pull against ours with docker image inspect. If a future latest behaves differently, that comparison is how you find out. The model checkpoint itself is pinned to a specific Hugging Face revision, and you download it on both units, not one: each rank loads the model from its own disk.
Why we released it as a script and not a dashboard
A two-unit cluster has exactly two machines, one cable, and one model. A script you run twice is honest about that topology. Every abstraction layer we could have added, a queue, a scheduler, a web UI, would have hidden the one thing you need to see when something goes wrong: which rank failed to reach the other over the cable. The README and the launch script name their variables plainly. CABLE_SUBNET is required on every run because the script builds the RDMA transport on a specific subnet and will not guess it. IFACE and HCA override the cable interface name and the RDMA device name, and you discover both on your own units with ip -br link and ibv_devices. Those two commands tell you what your hardware calls the cable; the script refuses to assume.
Stopping is symmetric and simple: docker rm -f qwen38-flash-next on both units. The launch script also saves the previous container's logs before replacing it, which matters more than it sounds. When a launch fails, the previous container's log is often the only record of why, and replacing the container without saving it would erase that record.
The hardware and the cable
Each DGX Spark is a GB10 system: one unit holds half of a tensor-parallel engine, and the cable between them carries the traffic that tensor parallelism generates. The recipe needs one direct QSFP cable between the two units' ConnectX-7 ports. Not through a switch, not through your office network: a single direct link. That constraint is not aesthetic. Tensor parallelism exchanges data between the units on every layer, and NCCL over RDMA (RoCE v2) on one HCA keeps that traffic on a wire that carries nothing else.
We sell the cable this recipe was measured with: a 0.5 m QSFP112 400G DAC cable for DGX Spark and other GB10 systems, $159, on our DGX Spark cluster cable page. We also keep cable pages for the other GB10 systems: the Dell Pro Max GB10, ASUS Ascent GX10, HP ZGX Nano, Lenovo ThinkStation PGX and MSI EdgeXpert. Two of our free repositories support the physical layer: gb10-cluster-guide walks cabling and network setup, and gb10-cluster-check verifies the link before you attempt a model launch. Run the checker first, so a cable problem shows up before you start a model.
If you want to understand what 400G on that link really delivers, our earlier post on DGX Spark cluster bandwidth goes deeper on what that link carries. And if you are still deciding whether two Sparks are the right hardware at all, our DGX Spark overview and our comparison of DGX Spark against Strix Halo and other local LLM hardware lay out the alternatives.
The configuration, line by line, and why each setting is there
The launch script starts vLLM with a specific set of flags. None of them is decoration, so it is worth understanding what each one buys and what it costs.
Tensor parallel 2 with expert parallelism
--tensor-parallel-size 2 --enable-expert-parallel puts one rank on each unit. Tensor parallelism 2 means every layer's work is split across the two ranks, so the two units must exchange activation data on every layer, which is why the cable and the NCCL-over-RDMA transport matter so much. Expert parallelism applies to the mixture-of-experts structure of the model: it spreads the MoE experts across the two ranks, so each unit holds its own share of the experts. This is the layout we measured; we did not measure the same pair without it.
Multi-token prediction at 4 tokens
MTP 4 tokens enables the model's own multi-token prediction head. vLLM uses that head to draft several tokens per decode step, and the setting here drafts 4. We measured the pair with this setting on; the decode numbers later in this post include its effect.
Concurrency and memory: 64 sequences at 75 percent
--max-num-seqs 64 and --gpu-memory-utilization 0.75 set the concurrency ceiling and the memory budget. These two numbers were not tuned per workload; they are the configuration we measured and the one the recipe ships.
Context length: 131,072 served, 262,144 native
--max-model-len 131072 serves half the model's native 262,144-token context. This is the setting people ask about most, so here is the reasoning. At 131,072 the two-unit KV pool holds 1,277,645 tokens, room for 9.8 full-length requests at once. Doubling the context length would mean fewer full-length requests fit in the same pool. Many workloads, such as retrieval pipelines and agentic loops, rarely send a single prompt that long, so the recipe spends the memory on concurrency. If your workload genuinely needs more than 131,072 tokens of context in one request, that is a conversation to have with our team before you buy, not a flag to change after.
Enforced eager mode
--enforce-eager disables CUDA graphs. We ran the configuration with eager execution because that is the configuration we measured; CUDA graphs were not part of this run. We are not claiming eager is faster. We are claiming that every number in this post was produced with this flag, and if you remove it you are running a different configuration than the one we measured, which means our numbers stop being your baseline. Reproducibility beats a benchmark number you cannot reproduce.
NCCL over RDMA on the cable
NCCL_NET=IB, RoCE v2, one HCA: these settings pin NCCL, the communication library tensor parallelism runs on, to the RDMA transport on the direct cable. Without them NCCL may choose a different network path, and the numbers in this post were measured with RDMA on the cable. The recipe pins the transport so you run the path we measured. This is also the setting most likely to need your intervention, which is why IFACE and HCA exist: your units may name the cable interface and the RDMA device differently than ours did, and the script will not guess.
Run order and the first-launch procedure
The order matters and the script encodes it: start unit B first, as rank 1, headless, serving no API; then start unit A as rank 0, which serves the OpenAI-compatible API on port 8888; then run the smoke test. Rank 1 runs headless and serves nothing on its own; rank 0 is the unit that answers API requests once both ranks are up. Follow the order the README gives: B first, A second.
Download the checkpoint and pull the image on both units before the first launch. When something does fail, read the saved previous-container logs on both units before touching anything: the launch script keeps them precisely so that the failure you just caused is still visible after the container that caused it is gone.
The settings most likely to need changing: the cable names
The script ships with the cable interface and RDMA device names a DGX Spark usually has. If your units name them differently, the two ranks cannot reach each other over the cable. The fix is simple: run ip -br link on each unit to see what the cable interface is actually called, run ibv_devices to list the RDMA devices, then set IFACE and HCA to match. CABLE_SUBNET is required and must be a subnet the cable actually carries. Verify the link with gb10-cluster-check before launching the model.
The second setting to think about is security. vLLM binds 0.0.0.0 with no API key. That is fine inside a private network and dangerous anywhere else. Keep port 8888 on a private network or behind an authenticating proxy, and treat the API as unauthenticated until something in front of it authenticates. This is a one-line deployment decision and it is the difference between a lab and an incident.
What we measured, and what the numbers mean
All numbers below come from our own DGX Spark pod on 2026-10-02, with both units' GPU clocks locked to a 300 to 2300 MHz range for the whole program so runs are comparable. Before any speed was recorded, a correctness gate passed 8 of 8; a speed number from a misconfigured engine is worse than no number, so the gate runs first on every configuration we measure. Our benchmark methodology page explains the full discipline, and the LLM benchmarks hub holds every model we have run this way.
Single-user decode
One user sees 35.2 tok/s on prose and 52.6 tok/s on code. In practice, 35 tok/s is faster than most people read, and the pair measured faster still on code. For interactive work, coding assistants, document drafting, this is the regime where the two-unit pair lives most comfortably.
Concurrency and the 10 tok/s floor
At 32 users at once, the pair aggregates 329.6 tok/s. We hold a per-user speed floor of 10 tok/s, below which interactive use starts to feel sluggish, and 32 users is the most the pair serves at that floor. The engine accepts up to 64 sequences, but 32 is the most it served while every user stayed above the floor. If your definition of acceptable is different, say 5 tok/s for background summarization, the practical user count is higher; we do not publish a number we did not measure at a floor we tested, so we leave that to your own pilot.
Prefill: nearly flat across prompt sizes
Prefill measured 3,089 tok/s at an 8K-token prompt and 3,069 tok/s at 32K. That flatness is the interesting part. A system whose prefill degrades as prompts grow will feel fast in a demo and slow in production, because production prompts are the long ones. Here, ingesting a 32,000-token document takes about the same per-token cost as ingesting an 8,000-token one, which makes long-document workflows predictable. If you feed the model retrieval contexts, contract sets, or codebases, flat prefill is the property that keeps response time from surprising you.
Long context is tested, not claimed
Needle retrieval passed at every tested length up to 120,001 tokens. That matters because a context window you cannot actually use is a spec-sheet fiction: a model that serves 131,072 tokens but loses a fact stated at position 90,000 is not serving you 131,072 useful tokens. The 120,001-token length is the longest our program tested, just short of the served window.
Energy: the number that makes two units the sweet spot
The pair drew 251 W at 1 user and 285 W at the 32-user ceiling while delivering 1.16 tokens per joule. That efficiency figure is why we call two units the most efficient layout per joule of the configurations we measured. A cluster whose wall power at full serving load is 285 W for the pair fits an office, not a server room.
Quality: the gate that matters most
Speed and efficiency are meaningless if the answers are wrong, so we run a quality battery alongside the speed program. On our retrieval-grounded CMMC battery, an agentic evaluation with strict scoring, the two-unit NVFP4 setup scored 0.978. That ties the much larger FP8 Qwen3.8-27B running on one RTX PRO 6000 Blackwell GPU, which scored 0.971. The per-model details live on our Qwen3.8-Flash-Next benchmark page.
What more units buy you
The same image and settings scale past two units, and we measured both bigger layouts with the same program. Four units as one engine (TP4) lift single-user decode to 41.4 tok/s prose and 61.1 code, reach 482.4 tok/s at 32 users, and serve 64 users at the floor while aggregating 648.7 tok/s. Prefill rises to 3,571 tok/s at 8K and 3,530 at 32K. Energy efficiency drops to 0.90 tokens per joule at 32 users, though it recovers to 1.18 at 64 users. A pod of four MSI EdgeXpert GB10 units matched the DGX Sparks within a few percent, which means the recipe is not chained to one vendor's chassis.
Eight units as one engine (TP8) push single-user decode to 45.7 tok/s prose and 63.5 code, with 604.7 tok/s at 32 users and 64 users at 769.7 tok/s, but efficiency drops to 0.58 tokens per joule at 32 users. Two separate TP4 groups of the same model gave about 1,266 tok/s combined across 64 users per group. One 8-way engine mainly buys single-user speed and one big shared context pool, not throughput; two independent four-unit groups buy throughput.
So the decision rule we give clients is simple. Two units is the most efficient layout per joule we measured for this model. Four units roughly double the user ceiling and stay reasonably efficient. Eight units as one engine is for the workload that needs one user to go as fast as possible, or one enormous shared context pool, and accepts the efficiency cost. Scaling is a capacity decision, not a default, and our private AI solutions practice helps clients pick the layout their actual workload needs rather than the biggest one they can afford.
Honest limits of this release
Three limits, stated plainly. First, the API surface: vLLM binds 0.0.0.0 with no API key, so port 8888 must stay on a private network or behind an authenticating proxy. Second, the script is two units only; the four- and eight-unit results above come from the same image and settings measured under the same program, but the released launch script automates the two-unit case. Third, and most important for calibrating your expectations: this release was not re-run for publication because both units were serving other models at the time, so every number in this post is from the 2026-10-02 run. A benchmark whose date you cannot see is a benchmark you cannot trust. The clocks were locked for the whole program, so the runs are internally comparable, but a fresh run on your units may differ for reasons of your own environment, and that is what the smoke test and your own pilot are for.
Where this fits in a private AI deployment
A two-unit Spark pod serving Qwen3.8-Flash-Next is one concrete answer to a common question: how do we run a capable model on hardware we own, inside our walls? This configuration does it at 285 W at full load, with a model whose quality on our retrieval-grounded CMMC battery matched a much larger model on a single datacenter GPU. It is not the answer for every workload, and our on-premise AI work starts by measuring the workload before recommending any hardware. But for teams that need long-context retrieval, code assistance, and document work at interactive speed on owned hardware, the two-unit recipe is the configuration we would deploy first.
Related reading
- DGX Spark Cluster Cable (QSFP112 400G DAC)
- DGX Spark Cluster: 4 to 8 Nodes, Switch vs Ring
- What Cable Do I Need to Connect Two DGX Sparks?
- DGX Spark 64GB: Specs, Price, and the Cable to Pair Two
- From Two DGX Sparks to a Switched Fabric: Cluster Topology
FAQ
Do I need exactly two DGX Spark units, or does the recipe work on other GB10 machines?
The launch script serves two units, but the configuration is not limited to the NVIDIA DGX Spark chassis. We measured that a pod of four MSI EdgeXpert GB10 units matched the DGX Sparks within a few percent with the same image and settings. Brand cable pages exist for the Dell Pro Max GB10, ASUS Ascent GX10, HP ZGX Nano, Lenovo ThinkStation PGX and MSI EdgeXpert.
Why does the recipe serve 131,072 tokens of context when the model supports 262,144?
The model's native max context is 262,144 tokens. At 131,072 the two-unit KV pool holds 1,277,645 tokens, room for 9.8 full-length requests at once. Serving the full native length would shrink how many long requests the pool can hold at once, so the recipe spends the memory on concurrency instead.
What does the direct QSFP cable actually do in this setup?
Tensor-parallel inference sends activation data between the two ranks on every layer. The recipe puts NCCL over RDMA (RoCE v2) on one ConnectX-7 port so that traffic stays on the direct link between the units instead of competing with your office network. Every number in this post was measured with that direct link.
How fast is Qwen3.8-Flash-Next on two DGX Sparks?
On the 2026-10-02 run: one user sees 35.2 tok/s prose and 52.6 tok/s code; 32 users at once aggregate 329.6 tok/s; and 32 users is the most the pair serves at our 10 tok/s per-user floor. Prefill measured 3,089 tok/s at an 8K-token prompt and 3,069 tok/s at 32K.
Is the API endpoint secure by default?
No. vLLM binds 0.0.0.0 with no API key. Keep port 8888 on a private network or behind an authenticating proxy before you let anyone else reach it.
Can I start at two units and add more later?
Yes, and the same program measured both bigger layouts. Four units (TP4) roughly double the user ceiling to 64 at the floor and reach 482.4 tok/s at 32 users. Eight units as one engine mainly buy single-user speed (45.7 tok/s prose) and one big context pool, not tokens per watt. Two units is the most efficient layout per joule at 1.16 tokens per joule.
What does the whole two-unit system cost to run in power terms?
The pair drew 251 W at 1 user and 285 W at the 32-user ceiling while serving 1.16 tokens per joule. Both units had GPU clocks locked to a 300 to 2300 MHz range during the measurement, so the numbers are comparable run to run.
What do I do if the two units cannot see each other over the cable?
Run ip -br link to see the cable interface name and ibv_devices to list RDMA devices, then set IFACE and HCA in the launch environment to what your units actually show. The CABLE_SUBNET variable is required on every run. Our gb10-cluster-check repository verifies the link before you start the model.
Get help with your deployment
Petronella Technology Group, Inc. has designed and run private AI on hardware our clients own since 2002, from our base in Raleigh, NC, including for teams working toward CMMC compliance. If you are planning a GB10 cluster, want this recipe deployed and verified on your units, or need help choosing between two, four and eight units for your workload, call Penny at 919-348-4912, or use the contact form and we will come back to you. We also help teams wire the model into real workflows, retrieval pipelines, agentic tools, compliance-sensitive document handling, which is where the difference between a working demo and a working system shows up. The cable, the cluster guide and the check tool are the start; the deployment is where we do our best work.
Reference build sheets, a hardening checklist for open-weight model hosting and a data-sovereignty map for running AI on your own hardware.
Get the free guide
