Benchmark · DGX Spark / GB10 · Other · Ryzen AI Max+ (Strix Halo)

Inkling 2 Minibenchmark results

This page holds every number we measured for this model: single-user prose speed of 158.2 tok/s on 1x RTX 5090 32 GB · native NVFP4/MXFP8 and aggregate throughput up to 208 tok/s on 1x GB10 · native NVFP4/MXFP8.

Last measured 2026-11-21
native NVFP4/MXFP8UD-Q4_K_XLvLLM inkling-2-mini image, DSpark 7vLLM inkling-2-mini image, no drafterllama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3llama.cpp Unsloth b12588 (Vulkan RADV), no drafter

Real throughput, latency and accuracy numbers, measured by Petronella Technology Group, Inc. on our own fleet. Last measured 2026-11-21.

Every number traced to a source file|No vendor marketing figures
158.2 tok/s
best single-user speed
4
measured configurations
Model card

At a glance

Hardware needed1x GB10 (msi3), 1x RTX 5090 32 GB (c1), 1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT)
EnginevLLM inkling-2-mini image, DSpark 7, vLLM inkling-2-mini image, no drafter, llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3, llama.cpp Unsloth b12588 (Vulkan RADV), no drafter
Quantizationnative NVFP4/MXFP8, UD-Q4_K_XL
Best single-user speed158.2 tok/s
Best aggregate (8 users)not measured
Verified contextnot verified
The fastest single-user prose speed we measured for this model is 158.2 tokens per second.

How speed holds up as users are added

Configuration1 user4 users16 users
1x GB10 · native NVFP4/MXFP837 tok/s80 tok/s
1x RTX 5090 32 GB · native NVFP4/MXFP8158.2 tok/s369 tok/s312 tok/s
1x ai11 · UD-Q4_K_XL · llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 330.7 tok/s30 tok/s
1x ai11 · UD-Q4_K_XL · llama.cpp Unsloth b12588 (Vulkan RADV), no drafter23.9 tok/s53 tok/s

Aggregate tokens per second across all users at once. One user is a single stream.

Prefill by prompt length

8K prompt32K prompt
1x GB10 · native NVFP4/MXFP82,358 tok/s2,410 tok/s
1x ai11 · UD-Q4_K_XL · llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3264 tok/s111 tok/s
1x ai11 · UD-Q4_K_XL · llama.cpp Unsloth b12588 (Vulkan RADV), no drafter361 tok/s258 tok/s
Measured numbers

Every measured number

Speed (single user)

Configurationdecode 1u prosedecode 1u code
1x GB10 (msi3) · vLLM inkling-2-mini image, DSpark 7 · native NVFP4/MXFP837 tok/s55.7 tok/s
1x RTX 5090 32 GB (c1) · vLLM inkling-2-mini image, no drafter · native NVFP4/MXFP8158.2 tok/s157.9 tok/s
1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3 · UD-Q4_K_XL30.7 tok/s31.4 tok/s
1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Unsloth b12588 (Vulkan RADV), no drafter · UD-Q4_K_XL23.9 tok/s23.8 tok/s

Source: ai11/RESULTS-2026-11-21-gorgon-halo-latest-stack.md

Aggregate throughput

Configurationpeak agg decode
1x GB10 (msi3) · vLLM inkling-2-mini image, DSpark 7 · native NVFP4/MXFP8208 tok/s

Source: ai11/RESULTS-2026-11-21-gorgon-halo-latest-stack.md

Prefill and latency

Configurationprefill 8kprefill 32k
1x GB10 (msi3) · vLLM inkling-2-mini image, DSpark 7 · native NVFP4/MXFP82,358 tok/s2,410 tok/s
1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3 · UD-Q4_K_XL264 tok/s111 tok/s
1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Unsloth b12588 (Vulkan RADV), no drafter · UD-Q4_K_XL361 tok/s258 tok/s

Source: ai11/RESULTS-2026-11-21-gorgon-halo-latest-stack.md

Other measurements

ConfigurationAggregate decode at 4 concurrent streamsTool-call accuracy on the 88-case corpusCoding score, 22 tasks, median of 3 runsResearch cases passed of 27 (1 run)Aggregate decode at 16 concurrent streamsUbuntu shell tasks score, 24 tasksNetwork device CLI tasks score, 14 tasksCoding score, 22 tasks (first of 2 runs)
1x GB10 (msi3) · vLLM inkling-2-mini image, DSpark 7 · native NVFP4/MXFP880 tok/s71.2 %0.968 score22 of 27
1x RTX 5090 32 GB (c1) · vLLM inkling-2-mini image, no drafter · native NVFP4/MXFP8369 tok/s73.5 %0.982 score22 of 27312 tok/s0.86 score0.806 score
1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3 · UD-Q4_K_XL30 tok/s
1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT) · llama.cpp Unsloth b12588 (Vulkan RADV), no drafter · UD-Q4_K_XL53 tok/s73.1 %18 of 270.99 score

Source: ai11/RESULTS-2026-11-21-gorgon-halo-latest-stack.md

Review

What it is

Inkling 2 Mini is a 26B mixture-of-experts model from Thinking Machines, released in November 2026. It is the small member of the Inkling 2 family, sized to fit on a single workstation graphics card or a single compact AI desktop, and it accepts images as well as text. It reasons before it answers by default, and it ships with its own small drafter model, which the vendor calls DSpark. The drafter guesses several tokens ahead and the main model checks the guesses in one pass, so a correct guess costs far less than generating each token on its own.

Petronella Technology Group, Inc. measured it the day after release on three machines we own: one DGX Spark class unit (GB10), one RTX 5090 workstation card, and one AMD Ryzen AI Max+ PRO 495 desktop. Every machine was updated to the newest kernel, driver and serving engine that morning, because a model this new depends on software that changes daily.

The headline finding is that this model's speed depends more on the serving engine than on the hardware. Where the vendor's own vLLM image and the native checkpoint run, it is quick. Where only the community GGUF build under llama.cpp runs, the same model is several times slower on hardware that is otherwise competitive. On quality it was the strongest coder of the three similar-sized models we ran on the same card, and the weakest of the three at tool calling. Both halves of that sentence matter before you deploy it.

Where it runs best

The RTX 5090 is the fastest place we have run it. With the native checkpoint under vLLM, one user gets 158.2 tokens per second of prose and 157.9 of code. The model fits in the card's memory with room for a working context, and four simultaneous requests share 369 tokens per second. This was without the drafter, which does not yet build for RTX 50-series cards, so the number has room to rise.

One GB10 unit is the best balanced host. The native checkpoint and the drafter both run there, single-user code generation reaches 55.7 tokens per second, and the unit's large unified memory let us serve it with a very long context window. It also reads long prompts quickly and keeps scaling as users are added: sixteen simultaneous requests shared 208 tokens per second, the highest aggregate we measured on that unit for this model.

The AMD desktop is the slowest of the three today, and the reason is software. No tuned engine for this model exists on AMD graphics yet, so we ran the Unsloth four-bit GGUF build under llama.cpp. That works and the answers are good, but single-user generation sits near thirty tokens per second and long prompts are read slowly. We expect this to improve as the AMD engines that already serve other models add support.

Measured speed

On the RTX 5090 under vLLM with the native checkpoint and no drafter, prose ran at 158.2 tokens per second and code at 157.9. Four streams together produced 369 tokens per second. At sixteen streams the aggregate was 312, lower than at four, which tells us the card was already saturated by this model at the smaller count and that the drafter's absence matters more as load rises. This is a single speed run, so treat it as a good first reading and not a final figure.

On one GB10 with vLLM, the native checkpoint and the drafter at depth seven, code ran at 55.7 tokens per second and prose at 37.0. The gap between the two is the drafter at work: code is predictable, so more of its guesses are accepted. Four streams produced eighty tokens per second together and sixteen streams 208. Prefill, the speed at which the model reads a prompt before it starts writing, was 2,358 tokens per second on the shorter prompt and 2,410 on the longer one. These are medians of three runs.

On the AMD desktop there are two useful shapes. With the ROCm build of llama.cpp and the drafter at depth three, one user gets 31.4 tokens per second of code and 30.7 of prose, but four streams together manage only thirty and prefill is 264 tokens per second on the shorter prompt and 111 on the longer one. With the Vulkan build and no drafter, one user gets 23.8 tokens per second of code and 23.9 of prose, four streams reach fifty-three, and prefill is 361 and 258. So the drafter build is the choice for one person typing short questions, and the Vulkan build is the better choice for several people or for long documents. Both are medians of three runs.

We tried the drafter at other depths on the AMD desktop and depth three was the best. Deeper drafts were slower on prose, and the drafter under Vulkan was slower than no drafter at all.

Quality and accuracy

We scored the model on our coding tasks, our research and reasoning cases, and our tool-calling corpus, using the vendor's recommended sampling and an outside judge model from a different family. These are not leaderboard rows. Our leaderboard uses fixed sampling and a fixed judge so that rows stay comparable over time, and these runs used neither.

Coding is the strength. On the RTX 5090 the median score over three runs of our coding tasks was 0.982, and on the GB10 it was 0.968. On the same RTX 5090, with the same judge and the same day, it scored above the two similar-sized models we compared it with on those tasks. The four-bit GGUF build on the AMD desktop scored 0.990 on its first run, which says the community quantization has not cost it coding ability.

It is solid on operations work too. On the RTX 5090 it scored 0.860 on our Ubuntu shell tasks and 0.806 on our network device command tasks.

Tool calling is the weakness. On our 88-case corpus it scored 73.5 percent on the RTX 5090, 71.2 percent on the GB10 and 73.1 percent on the AMD desktop. The misses include invented calls: asked something no tool could answer, it sometimes called a tool anyway. The two comparison models on the same card were clearly better on this corpus. If the job is an agent that must know when not to act, test this carefully before you rely on it.

Research and reasoning was mixed. It passed 22 of 27 cases on the GB10 and on its first RTX 5090 run, and 18 on the second RTX 5090 run, so expect some spread between runs at this sampling. On the AMD desktop it passed 18, but five requests ran out of time at that machine's lower speed, so that figure reflects serving speed as much as model quality.

In our agent harness tests, three small coding jobs each run through three different open source coding agents, every job passed on both the GB10 and the AMD desktop. Those jobs are easy on purpose. They show that the model's tool format works end to end with common agents, and they do not rank quality.

How we ran it

Every machine was fully updated on the morning of the test: operating system packages, kernel, graphics driver and serving engine. We then stamped those versions into the result files. One client on our network ran the same speed test against each server: a fixed prose prompt and a fixed code prompt for single-user speed, several streams at once for aggregate throughput, and fresh long prompts for prefill. Reasoning tokens count as output.

On the GB10 and the RTX 5090 we used the vLLM project's own image for this model with the native checkpoint, which mixes four-bit and eight-bit floating point weights. On the GB10 the drafter ran at depth seven. On the RTX 5090 the drafter was off because it does not build for that card family yet. On the AMD desktop we used the Unsloth four-bit GGUF with the Unsloth build of llama.cpp, once on ROCm with the drafter and once on Vulkan without it.

The GB10 and AMD speed rows are medians of three runs. The RTX 5090 speed row is one run. Coding ran three times on the GB10 and the RTX 5090 and twice on the AMD desktop. Research ran once on the GB10 and the AMD desktop and twice on the RTX 5090. Agent harness jobs ran inside a sandboxed container, never directly on a host.

When to pick it

Pick it when the main job is writing and fixing code on one card or one compact desktop, and when you can run the vendor's vLLM image. It was the best coder in its size class in our same-card comparison and it is fast where its native engine runs.

Pick the GB10 for it when you need long context, long prompts and several users on one quiet box. Pick the RTX 5090 when one or a few users want the fastest responses and the context you need fits in the card.

Wait, or pick another model, when the work is tool calling with strict rules about when not to act, or when your only hardware is an AMD desktop and speed matters. On that desktop a different model with a tuned engine is currently more than twice as fast for one user.

How each configuration was served

ConfigurationHardwareUnitsEngineQuantMeasuredNotes
1x GB10 · native NVFP4/MXFP81x GB10 (msi3)1vLLM inkling-2-mini image, DSpark 7native NVFP4/MXFP82026-11-21
1x RTX 5090 32 GB · native NVFP4/MXFP81x RTX 5090 32 GB (c1)1vLLM inkling-2-mini image, no drafternative NVFP4/MXFP82026-11-21
1x ai11 · UD-Q4_K_XL · llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 31x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT)1llama.cpp Unsloth b12588 (ROCm), DSpark drafter depth 3UD-Q4_K_XL2026-11-21
1x ai11 · UD-Q4_K_XL · llama.cpp Unsloth b12588 (Vulkan RADV), no drafter1x ai11 (Minisforum MS-S1 MAX, Ryzen AI Max+ PRO 495 / Radeon 8065S, 192 GB unified, 176 GiB GTT)1llama.cpp Unsloth b12588 (Vulkan RADV), no drafterUD-Q4_K_XL2026-11-21

FAQ

How fast is Inkling 2 Mini for a single user?
It depends on the hardware and the engine. On an RTX 5090 with the native checkpoint we measured 158.2 tokens per second of prose. On one GB10 with the drafter we measured 55.7 tokens per second of code and 37.0 of prose. On an AMD Ryzen AI Max+ PRO 495 desktop with the GGUF build and the drafter we measured about thirty-one.
Does Inkling 2 Mini fit on one RTX 5090?
Yes. The native checkpoint loaded and served on a single card in our lab with room for a working context. We ran it alongside the card's other light duties after freeing memory, and we tested it with up to sixteen simultaneous requests.
What is the DSpark drafter and is it worth using?
It is a small companion model that proposes several tokens ahead, which the main model then accepts or rejects in one pass. Where it runs well it is worth using: on the GB10 it is why code generation is so much faster than prose. On the AMD desktop it helps one user under ROCm, hurts under Vulkan, and lowers throughput when several people share the machine.
Why is it so much slower on the AMD desktop?
Because of the engine, not the chip. The same desktop runs another model of similar size much faster with an engine tuned for it. For Inkling 2 Mini only the general llama.cpp path exists on AMD graphics so far, and that path reads long prompts slowly. We will retest when a tuned engine adds support.
Is the four-bit GGUF build as accurate as the native checkpoint?
On coding and tool calling, yes within the spread between runs. The GGUF build scored 0.990 on its first coding run and 73.1 percent on tool calls, against 73.5 percent for the native checkpoint on the RTX 5090. Its lower research result came mostly from requests that timed out at the desktop's lower speed.
Is Inkling 2 Mini good at tool calling?
It was the weakest of the three similar-sized models we compared on one card, at 73.5 percent on our 88-case corpus, and it sometimes invented a tool call when it should have declined. It did complete every job in our agent harness tests, so the format works. For agents with real permissions, test the decline cases before you trust it.
How many users can one GB10 serve with this model?
In our test sixteen simultaneous requests shared 208 tokens per second and four shared eighty. Throughput was still rising at sixteen, so that is the largest count we ran, not a ceiling. Each user's own speed falls as users are added, so size it by the response time your people will accept.
Are these results on your leaderboard?
No. These runs used the vendor's recommended sampling and an outside judge reached through our gateway. Our leaderboard keeps a fixed judge and fixed sampling so rows stay comparable, and adding this model means running it again under those settings.
Can you test Inkling 2 Mini on our hardware?
Yes. Call Penny at 919-348-4912 and describe the machines you have, or use the contact form to ask for a benchmark. We run the same client and the same task sets we used here, on your hardware, and give you the result files.

Run this model on your own hardware

Call Penny at 919-348-4912 and describe your fleet, or ask us to benchmark a model on the hardware you already own.