Inkling 2 Minibenchmark results
This page holds every number we measured for this model: single-user prose speed of 158.2 tok/s on 1x RTX 5090 32 GB · native NVFP4/MXFP8 and aggregate throughput up to 208 tok/s on 1x GB10 · native NVFP4/MXFP8.
Last measured 2026-11-21Real throughput, latency and accuracy numbers, measured by Petronella Technology Group, Inc. on our own fleet. Last measured 2026-11-21.
At a glance
| Hardware needed | 1x GB10 (msi3), 1x RTX 5090 32 GB (c1), 1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) |
| Engine | vLLM inkling-2-mini image, DSpark 7, vLLM inkling-2-mini image, no drafter, llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3, llama.cpp Unsloth b12588 (Vulkan RADV), no drafter |
| Quantization | native NVFP4/MXFP8, UD-Q4_K_XL |
| Best single-user speed | 158.2 tok/s |
| Best aggregate (8 users) | not measured |
| Verified context | not verified |
How speed holds up as users are added
| Configuration | 1 user | 4 users | 16 users |
|---|---|---|---|
| 1x GB10 · native NVFP4/MXFP8 | 37 tok/s | 80 tok/s | |
| 1x RTX 5090 32 GB · native NVFP4/MXFP8 | 158.2 tok/s | 369 tok/s | 312 tok/s |
| 1x ai11 · UD-Q4_K_XL · llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3 | 30.7 tok/s | 30 tok/s | |
| 1x ai11 · UD-Q4_K_XL · llama.cpp Unsloth b12588 (Vulkan RADV), no drafter | 23.9 tok/s | 53 tok/s |
Aggregate tokens per second across all users at once. One user is a single stream.
Prefill by prompt length
| 8K prompt | 32K prompt | |
|---|---|---|
| 1x GB10 · native NVFP4/MXFP8 | 2,358 tok/s | 2,410 tok/s |
| 1x ai11 · UD-Q4_K_XL · llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3 | 264 tok/s | 111 tok/s |
| 1x ai11 · UD-Q4_K_XL · llama.cpp Unsloth b12588 (Vulkan RADV), no drafter | 361 tok/s | 258 tok/s |
Every measured number
Speed (single user)
| Configuration | decode 1u prose | decode 1u code |
|---|---|---|
| 1x GB10 (msi3) · vLLM inkling-2-mini image, DSpark 7 · native NVFP4/MXFP8 | 37 tok/s | 55.7 tok/s |
| 1x RTX 5090 32 GB (c1) · vLLM inkling-2-mini image, no drafter · native NVFP4/MXFP8 | 158.2 tok/s | 157.9 tok/s |
| 1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3 · UD-Q4_K_XL | 30.7 tok/s | 31.4 tok/s |
| 1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Unsloth b12588 (Vulkan RADV), no drafter · UD-Q4_K_XL | 23.9 tok/s | 23.8 tok/s |
Source: ai11/RESULTS-2026-11-21-gorgon-halo-latest-stack.md
Aggregate throughput
| Configuration | peak agg decode |
|---|---|
| 1x GB10 (msi3) · vLLM inkling-2-mini image, DSpark 7 · native NVFP4/MXFP8 | 208 tok/s |
Source: ai11/RESULTS-2026-11-21-gorgon-halo-latest-stack.md
Prefill and latency
| Configuration | prefill 8k | prefill 32k |
|---|---|---|
| 1x GB10 (msi3) · vLLM inkling-2-mini image, DSpark 7 · native NVFP4/MXFP8 | 2,358 tok/s | 2,410 tok/s |
| 1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3 · UD-Q4_K_XL | 264 tok/s | 111 tok/s |
| 1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Unsloth b12588 (Vulkan RADV), no drafter · UD-Q4_K_XL | 361 tok/s | 258 tok/s |
Source: ai11/RESULTS-2026-11-21-gorgon-halo-latest-stack.md
Other measurements
| Configuration | Aggregate decode at 4 concurrent streams | Tool-call accuracy on the 88-case corpus | Coding score, 22 tasks, median of 3 runs | Research cases passed of 27 (1 run) | Aggregate decode at 16 concurrent streams | Ubuntu shell tasks score, 24 tasks | Network device CLI tasks score, 14 tasks | Coding score, 22 tasks (first of 2 runs) |
|---|---|---|---|---|---|---|---|---|
| 1x GB10 (msi3) · vLLM inkling-2-mini image, DSpark 7 · native NVFP4/MXFP8 | 80 tok/s | 71.2 % | 0.968 score | 22 of 27 | ||||
| 1x RTX 5090 32 GB (c1) · vLLM inkling-2-mini image, no drafter · native NVFP4/MXFP8 | 369 tok/s | 73.5 % | 0.982 score | 22 of 27 | 312 tok/s | 0.86 score | 0.806 score | |
| 1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3 · UD-Q4_K_XL | 30 tok/s | |||||||
| 1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Unsloth b12588 (Vulkan RADV), no drafter · UD-Q4_K_XL | 53 tok/s | 73.1 % | 18 of 27 | 0.99 score |
Source: ai11/RESULTS-2026-11-21-gorgon-halo-latest-stack.md
What it is
Inkling 2 Mini is a 26B mixture-of-experts model from Thinking Machines, released in November 2026. It is the small member of the Inkling 2 family, sized to fit on a single workstation graphics card or a single compact AI desktop, and it accepts images as well as text. It reasons before it answers by default, and it ships with its own small drafter model, which the vendor calls DSpark. The drafter guesses several tokens ahead and the main model checks the guesses in one pass, so a correct guess costs far less than generating each token on its own.
Petronella Technology Group, Inc. measured it the day after release on three machines we own: one DGX Spark class unit (GB10), one RTX 5090 workstation card, and one AMD Ryzen AI Max+ PRO 495 desktop. Every machine was updated to the newest kernel, driver and serving engine that morning, because a model this new depends on software that changes daily.
The headline finding is that this model's speed depends more on the serving engine than on the hardware. Where the vendor's own vLLM image and the native checkpoint run, it is quick. Where only the community GGUF build under llama.cpp runs, the same model is several times slower on hardware that is otherwise competitive. On quality it was the strongest coder of the three similar-sized models we ran on the same card, and the weakest of the three at tool calling. Both halves of that sentence matter before you deploy it.
Where it runs best
The RTX 5090 is the fastest place we have run it. With the native checkpoint under vLLM, one user gets 158.2 tokens per second of prose and 157.9 of code. The model fits in the card's memory with room for a working context, and four simultaneous requests share 369 tokens per second. This was without the drafter, which does not yet build for RTX 50-series cards, so the number has room to rise.
One GB10 unit is the best balanced host. The native checkpoint and the drafter both run there, single-user code generation reaches 55.7 tokens per second, and the unit's large unified memory let us serve it with a very long context window. It also reads long prompts quickly and keeps scaling as users are added: sixteen simultaneous requests shared 208 tokens per second, the highest aggregate we measured on that unit for this model.
The AMD desktop is the slowest of the three today, and the reason is software. No tuned engine for this model exists on AMD graphics yet, so we ran the Unsloth four-bit GGUF build under llama.cpp. That works and the answers are good, but single-user generation sits near thirty tokens per second and long prompts are read slowly. We expect this to improve as the AMD engines that already serve other models add support.
Measured speed
On the RTX 5090 under vLLM with the native checkpoint and no drafter, prose ran at 158.2 tokens per second and code at 157.9. Four streams together produced 369 tokens per second. At sixteen streams the aggregate was 312, lower than at four, which tells us the card was already saturated by this model at the smaller count and that the drafter's absence matters more as load rises. This is a single speed run, so treat it as a good first reading and not a final figure.
On one GB10 with vLLM, the native checkpoint and the drafter at depth seven, code ran at 55.7 tokens per second and prose at 37.0. The gap between the two is the drafter at work: code is predictable, so more of its guesses are accepted. Four streams produced eighty tokens per second together and sixteen streams 208. Prefill, the speed at which the model reads a prompt before it starts writing, was 2,358 tokens per second on the shorter prompt and 2,410 on the longer one. These are medians of three runs.
On the AMD desktop there are two useful shapes. With the ROCm build of llama.cpp and the drafter at depth three, one user gets 31.4 tokens per second of code and 30.7 of prose, but four streams together manage only thirty and prefill is 264 tokens per second on the shorter prompt and 111 on the longer one. With the Vulkan build and no drafter, one user gets 23.8 tokens per second of code and 23.9 of prose, four streams reach fifty-three, and prefill is 361 and 258. So the drafter build is the choice for one person typing short questions, and the Vulkan build is the better choice for several people or for long documents. Both are medians of three runs.
We tried the drafter at other depths on the AMD desktop and depth three was the best. Deeper drafts were slower on prose, and the drafter under Vulkan was slower than no drafter at all.
Quality and accuracy
We scored the model on our coding tasks, our research and reasoning cases, and our tool-calling corpus, using the vendor's recommended sampling and an outside judge model from a different family. These are not leaderboard rows. Our leaderboard uses fixed sampling and a fixed judge so that rows stay comparable over time, and these runs used neither.
Coding is the strength. On the RTX 5090 the median score over three runs of our coding tasks was 0.982, and on the GB10 it was 0.968. On the same RTX 5090, with the same judge and the same day, it scored above the two similar-sized models we compared it with on those tasks. The four-bit GGUF build on the AMD desktop scored 0.990 on its first run, which says the community quantization has not cost it coding ability.
It is solid on operations work too. On the RTX 5090 it scored 0.860 on our Ubuntu shell tasks and 0.806 on our network device command tasks.
Tool calling is the weakness. On our 88-case corpus it scored 73.5 percent on the RTX 5090, 71.2 percent on the GB10 and 73.1 percent on the AMD desktop. The misses include invented calls: asked something no tool could answer, it sometimes called a tool anyway. The two comparison models on the same card were clearly better on this corpus. If the job is an agent that must know when not to act, test this carefully before you rely on it.
Research and reasoning was mixed. It passed 22 of 27 cases on the GB10 and on its first RTX 5090 run, and 18 on the second RTX 5090 run, so expect some spread between runs at this sampling. On the AMD desktop it passed 18, but five requests ran out of time at that machine's lower speed, so that figure reflects serving speed as much as model quality.
In our agent harness tests, three small coding jobs each run through three different open source coding agents, every job passed on both the GB10 and the AMD desktop. Those jobs are easy on purpose. They show that the model's tool format works end to end with common agents, and they do not rank quality.
How we ran it
Every machine was fully updated on the morning of the test: operating system packages, kernel, graphics driver and serving engine. We then stamped those versions into the result files. One client on our network ran the same speed test against each server: a fixed prose prompt and a fixed code prompt for single-user speed, several streams at once for aggregate throughput, and fresh long prompts for prefill. Reasoning tokens count as output.
On the GB10 and the RTX 5090 we used the vLLM project's own image for this model with the native checkpoint, which mixes four-bit and eight-bit floating point weights. On the GB10 the drafter ran at depth seven. On the RTX 5090 the drafter was off because it does not build for that card family yet. On the AMD desktop we used the Unsloth four-bit GGUF with the Unsloth build of llama.cpp, once on ROCm with the drafter and once on Vulkan without it.
The GB10 and AMD speed rows are medians of three runs. The RTX 5090 speed row is one run. Coding ran three times on the GB10 and the RTX 5090 and twice on the AMD desktop. Research ran once on the GB10 and the AMD desktop and twice on the RTX 5090. Agent harness jobs ran inside a sandboxed container, never directly on a host.
When to pick it
Pick it when the main job is writing and fixing code on one card or one compact desktop, and when you can run the vendor's vLLM image. It was the best coder in its size class in our same-card comparison and it is fast where its native engine runs.
Pick the GB10 for it when you need long context, long prompts and several users on one quiet box. Pick the RTX 5090 when one or a few users want the fastest responses and the context you need fits in the card.
Wait, or pick another model, when the work is tool calling with strict rules about when not to act, or when your only hardware is an AMD desktop and speed matters. On that desktop a different model with a tuned engine is currently more than twice as fast for one user.
How each configuration was served
| Configuration | Hardware | Units | Engine | Quant | Measured | Notes |
|---|---|---|---|---|---|---|
| 1x GB10 · native NVFP4/MXFP8 | 1x GB10 (msi3) | 1 | vLLM inkling-2-mini image, DSpark 7 | native NVFP4/MXFP8 | 2026-11-21 | |
| 1x RTX 5090 32 GB · native NVFP4/MXFP8 | 1x RTX 5090 32 GB (c1) | 1 | vLLM inkling-2-mini image, no drafter | native NVFP4/MXFP8 | 2026-11-21 | |
| 1x ai11 · UD-Q4_K_XL · llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3 | 1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) | 1 | llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3 | UD-Q4_K_XL | 2026-11-21 | |
| 1x ai11 · UD-Q4_K_XL · llama.cpp Unsloth b12588 (Vulkan RADV), no drafter | 1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) | 1 | llama.cpp Unsloth b12588 (Vulkan RADV), no drafter | UD-Q4_K_XL | 2026-11-21 |
FAQ
How fast is Inkling 2 Mini for a single user?
Does Inkling 2 Mini fit on one RTX 5090?
What is the DSpark drafter and is it worth using?
Why is it so much slower on the AMD desktop?
Is the four-bit GGUF build as accurate as the native checkpoint?
Is Inkling 2 Mini good at tool calling?
How many users can one GB10 serve with this model?
Are these results on your leaderboard?
Can you test Inkling 2 Mini on our hardware?
Run this model on your own hardware
Call Penny at 919-348-4912 and describe your fleet, or ask us to benchmark a model on the hardware you already own.