Self-Hosted LLM Benchmarks:The Private-AI Model Leaderboard Measured on Our Own GPUs
These self-hosted LLM benchmarks evaluate open-weight and cloud models the way a compliance-bound firm actually uses them: coding, hard debugging, cited research, agentic tool use and long-form writing, judged by a three-model frontier panel and run on the GPU fleet Petronella Technology Group, Inc. operates in Raleigh, so CUI and privileged data never touch a third party.
What These Self-Hosted LLM Benchmarks Measure and Why It Matters
These self-hosted LLM benchmarks measure how well a model does the work a regulated firm needs done, on hardware that firm can own, rather than how it scores on a public test set it may have been trained on. That distinction is the whole reason the page exists.
Public leaderboards rank models on academic corpora and, in most cases, republish numbers supplied by the vendor. A buyer who needs a model to fix a bug in a production script, draft a cited research memo or refuse an unsafe tool call gets little from a single aggregate score. So Petronella Technology Group, Inc. built its own harness. Each model runs a fixed corpus per capability: 22 core and 20 hard coding tasks drawn from real operations, SEO and debugging work; 27 research cases that cover summarization, multi-hop reasoning and citation accuracy; an 88-case tool decline-safety corpus; agentic multi-turn tasks; and a 3,000-word blog post scored against a 10-criteria rubric. Where a check can be deterministic (does the code run, are the citations right) it is, and a language-model judge covers the rest.
Three rules make the numbers trustworthy. First, every score is measured on the fleet Petronella Technology Group operates in Raleigh, never copied from a vendor. Second, no model is ever graded by a model from its own family, because self-judging inflates scores by 20 to 37 percent in our measurements. Third, every result carries its serving configuration, because a wrong tool parser or a temperature-0 run on an agentic coder looks identical to a weak model. Where a public benchmark is the right yardstick we run it ourselves: the SWE-bench Verified figures on this page come from the official SWE-bench Verified harness on the stratified 150-case set, with the sampling configuration published beside each result.
Why it matters: the models that win here are the models you can run inside your own compliance boundary. Coding and cited research on CUI, PHI or privileged material cannot be sent to a third-party API, so the practical question is not which model is best in the abstract but which model is best on the hardware tier you can justify. That is why the picks below are organised by tier, from a single NVIDIA GB10 appliance to a four-way H200 pool, and why every leaderboard row carries its latency, temperature and measurement date. Coding, agentic tool use, research and long-form writing are ranked separately, because the model that wins one rarely wins all four, and quantization format, speculative decoding and interconnect are treated as first-class variables rather than footnotes.
If you are still deciding whether to run models yourself, start with our guide on how to self-host an LLM, then read about private AI solutions for regulated firms to see how deployment, access control and data handling fit together. The rest of this page is the evidence: the latest study, the best model for each hardware tier, the category leaders, the methodology, fourteen findings from the last four months, a buyer's guide by tier, and the questions buyers ask most.
Need one of these models running on hardware you own? Schedule a 30-minute private-AI call with Petronella Technology Group, Inc.
Latest Study: GLM-5.3-Flash Replaces GLM-5.2 on the H200 Quad
GLM-5.3-Flash-FP8 is the new top pick on the four-way H200 pool: it beats GLM-5.2 on coding and long-form writing, ties on research, and answers a coding case in 4.4 seconds instead of 80.
GLM-5.3-Flash replaces GLM-5.2 on the H200 quad: better scores at a twentieth of the latency.
Measured the day it was released, under the same campaign judge and the same 22-case coding set as the model it replaced. The new 320B mixture-of-experts (18B active, MIT, 1M-token context) wins coding and long-form writing outright, ties on research, and answers a coding case in 4.4 seconds instead of 80. Two traps worth knowing before you deploy it are in the findings below.
- Coding 0.987 (n=3), wins
- Blog 0.889 and research 0.852
- 4.4 s per coding case, 233 to 339 t/s decode
- Coding 0.964 (n=1)
- Blog 0.833 and research 0.852
- 80 s per coding case, 15 t/s on the blog run
Previous featured study (July 15): Gemma-4-26B-A4B vs the Qwen3.6 family, where task difficulty decided the winner. It remains in the findings below.
Best Model for Your Hardware
The best self-hosted model is decided by the memory pool and the interconnect you have, so the picks are organised by hardware tier rather than as one ranking. Our smallest tier is a 128 GB appliance, larger than the consumer-VRAM classes most guides sort by, and each pick names the model, the quantization format and the trade-offs that decided it.
--tool-call-parser mistral.Want one of these running inside your compliance boundary?
Petronella Technology Group, Inc. designs, builds and operates private AI on hardware you own, from a single GB10 appliance to a multi-H200 enclave, for firms that handle CUI, PHI or privileged data. Every pick above is a configuration we run ourselves.
Who Tops Each Task
Claude Opus-4.7 (cloud) and Laguna-S-2.1-NVFP4 (local) share the top of the core coding set at 0.991, Ornith-1.0-397B-FP8 holds the hard set at 0.971, three local models tie on research at 0.963, and Qwen3.6-27B dense leads blog writing at 1.000. Best score per model, cross-judged (generator never grades itself). Cloud models are shown as reference lines; everything marked LOCAL runs entirely on the Petronella Technology Group fleet.
Coding, core
Coding, hard set
Research & reasoning
Blog writing
Full tables with per-rep ranges, latency, temperature and dates
How the Scores Are Produced: Cross-Judging and the Judge Lock
Every score comes from a fixed task corpus graded by a judge that never shares a model family with the generator. Headline studies use a three-judge frontier panel median; longitudinal leaderboard rows use a locked campaign judge so a row measured in May is comparable to one measured in August.
Task-appropriate, cross-judged. Each model runs a fixed corpus per capability: 22 core + 20 hard coding tasks, 27 research cases, an 88-case tool decline-safety corpus, agentic multi-turn tasks, and a 3,000-word blog with a 10-criteria rubric. Coding and research combine deterministic auto-checks (does the code run, are the citations right) with an LM judge; blog combines structural gates with a judge. The generator is never the judge, and never shares its model family with the judge.
Frontier judge panel. Headline comparisons are graded by three independent frontier models, GPT-5.4 (OpenAI), Claude Opus-4.8 (Anthropic) and DeepSeek-V4 (GLM-5.2 held that seat until August 2026), scoring identical saved generations, with the median taken per case so no single judge's bias can decide a result. DeepSeek-V4-Flash also serves as a validated free grader for bulk work.
Judge lock for longitudinal rows. On the leaderboards, the campaign judge is locked to GPT-4.1 (Session-13/15 lock-in; Haiku-4.5 or validated-equivalent DeepSeek-V4-Flash on legacy rows), so scores measured months apart stay comparable. Each row also records the number of reps (n=), the sampling temperature and the Measured date, and the tables are regenerated from the harness result files rather than edited by hand. Headline studies on this page use the three-judge panel median.
Serving stack, recorded with every result. Local models are served with vLLM in NVFP4, FP8 or int4. Speculative decoding (multi-token prediction and block-draft heads) and tool-call parsers are set per model and published beside the score, because the wrong flag changes the result more than the model does; the vLLM documentation on speculative decoding and tool calling describes the flags referenced in the findings.
Measured on-fleet. Everything is timed and scored on the GPUs Petronella Technology Group, Inc. owns: the point is that a private, compliance-bound firm can run capable AI on privileged and CUI data without sending a single token to a third party. No number on this page is vendor-supplied.
What We Have Learned
Fourteen findings from May to August 2026, newest first, are the headline results driving the private-AI deployment decisions of Petronella Technology Group, Inc.: every number measured on our own hardware with a real evaluation harness, not vendor marketing.
Four GB10 appliances on a 200G RoCE switch run one 512 GB model pool. The fabric is not the limit; memory bandwidth still is.
We moved four NVIDIA GB10 units onto a switched 200G RoCE fabric (two MikroTik CRS812 switches, 196 Gb/s node-to-node RDMA measured through the switch with no PFC or ECN tuning) and ran the first 4-way models on it. Qwen3.8-Flash-Next (125B, 6B active, NVFP4) split four ways with expert parallelism decodes 40 to 77 t/s for one user and 167 to 184 t/s across six streams with a 2.6-million-token KV cache: 21 to 54% faster per stream and 40% more aggregate than the same model on a two-unit pair. GLM-5.3-Flash (18B active) on the same four units runs 26 to 47 t/s single-stream and 91 t/s aggregate. The smaller-active-parameter model wins on this bandwidth-bound platform, again.
What the sweep taught us. First: plain 4-way tensor parallelism cannot load this NVFP4 mixture-of-experts at all (the expert width does not divide four ways for the 4-bit kernels); expert parallelism is required, and using both 100G halves of each node's link adds a further 5 to 15%. Second: the speculative-decoding sweet spot is four multi-token-prediction drafts: six drafts and CUDA graphs both lost speed, and a block-draft head (DFlash2) that lifts GLM to 69 t/s on math for a single user halves throughput once six users share it. Third: one GB10 can serve the 125B model alone by memory-mapping its 51B n-gram table from NVMe (25 to 34 t/s), so four independent units deliver about 345 t/s aggregate against 184 for the 4-way pool: choose per-user speed and context, or aggregate. Quality evaluations on the fabric are next; these are throughput results.
GLM-5.3-Flash: a 320B model that runs like a 35B, and the new top pick for the H200 quad.
Released August 26 under MIT, 320B total with 18B active, native FP8, 1M-token context. On four NVLink H200s it prefills 20,600 tok/s and decodes 233 to 339 t/s (code to math) with a 2.86-million-token KV cache. Quality: coding 0.987 (n=3) under our campaign judge, and the three-judge frontier panel scores it 0.9955, tying the best of 51 panel rows; it leads the panel on research at 0.912. Two traps that dominate real use: thinking is on by default and gated by reasoning_effort rather than a Qwen-style toggle, so a default long-form request can spend its whole budget on invisible reasoning; and free-form summarization fabricates (0.736) even while citation accuracy is 0.991. It replaced GLM-5.2 on the fleet the same day.
SWE-bench Verified went from 67.3% to 75.3% by changing nothing but the sampling config.
DeepSeek-V4-Flash-0731 at the vendor's recommended sampling scored 75.3% (113/150) on the stratified SWE-bench Verified set on two H200s, tying GLM-5.2's 76.0%. The same model at temperature 0 collapsed to 2.7% usable patches. Our earlier 67.3% headline was a temperature-0 run. Lesson for anyone comparing agentic coders: the sampling configuration is part of the model, and any published number without it is not reproducible.
Block-draft speculation is the biggest single lever on GB10, and NVFP4 4-bit ties FP8 quality.
Block-draft speculative decoding (DSpark) took a Nemotron-3.5 mixture-of-experts on one GB10 from 75.1 to 184.0 t/s on math, 2.45x, beating standard multi-token prediction by 79% on the same hardware. It is not free for every model: against controls that change only the speculative flags, Nemotron loses about 1.8 points while Qwen3.6-35B and Qwen3.8-27B show no measurable cost. Separately, NVFP4 (4-bit weights and activations with the sensitive layers kept in BF16) statistically ties FP8 on the coding set under the three-judge panel (0.9795 vs 0.9762), while naive 4-bit formats of the same model lose 4 to 7 points. Format matters, not bit-width. And always state the content class: the same config measured 184 t/s on math, 121 on code and 105 on prose.
Cabling four GB10s together made everything slower. A switch fixed it. Plus: a GB10 is a prefill machine.
Direct-attached, four GB10 units in tensor-parallel decoded a 27B dense model at 3.5 t/s versus 8.3 on one unit, because all-to-all RDMA is physically impossible with two ports per node: a 4-way cluster needs a switch, not cables. One of the four ConnectX-7 cards was also firmware-throttled to 12.7 Gb/s and reported healthy; only a bandwidth test found it (111.9 Gb/s after a firmware update). Two weeks later the same four units on a 200G RoCE switch delivered the fabric results above. The other durable number: on one GB10, prefill runs at 1,963 tok/s while dense decode runs at 8.3 t/s, a 237:1 ratio within 12% of the LPDDR5X bandwidth prediction. Routing rule: long-prompt, short-output work (RAG, summarization, classification, reranking) belongs on the GB10; long generations belong on the Blackwell workstation card.
Mainline llama.cpp now does multi-token-prediction drafting: about 2x for free on GB10 GGUFs.
The forum claim of ~27 t/s for a Q4_K_XL GGUF on a GB10 was real: it used llama.cpp's newly merged MTP speculative decoding (--spec-type draft-mtp --spec-draft-n-max 3). With it, llama.cpp Q4_K_XL plus MTP reaches 20 to 25 t/s on a dense model where vLLM FP8 plus MTP manages 12.2 on the same unit. Trap: the draft-dspark flag on a plain GGUF is a silent no-op, a healthy server at baseline speed. Verify the creating MTP draft context log line before believing any speculative flag. The engine comparison behind this is written up in Ollama versus vLLM on Blackwell.
One datacenter Blackwell card handles about 4.5x the concurrent load of a GB10 appliance, and keeps scaling to 128 streams.
Serving many users at once is where the hardware tiers separate. On an identical model (Gemma-4-26B-A4B, NVFP4), a single RTX PRO 6000 Blackwell serves 2,480 tok/s across 32 concurrent streams to the NVIDIA GB10 appliance's 554, about 4.5x. And the datacenter card keeps scaling where the bandwidth-bound appliance can't: on a dense 27B it reaches ~2,790 tok/s at 128 concurrent sessions with zero preemptions (memory-bound, not compute-bound), while the GB10 plateaus by ~32. Reach for the Blackwell card for multi-user and bursty serving; the GB10 for single-stream appliance and edge. The full run is in RTX PRO 6000 versus GB10 head-to-head.
We swept the whole ladder on one model: appliance to 4x NVLink H200. More GPUs buy model size and users, not single-user speed.
Same model (Qwen3.6-35B-A3B, FP8), same engine and settings, swept to saturation on every hardware tier we deploy. Single-user speed barely moves across the top four tiers (~200 to 216 t/s). What scales is capacity: aggregate throughput, comfortable concurrent users, and the memory pool that determines the largest private model you can run.
| Hardware | Memory pool | 1 user | Peak aggregate | Comfortable users |
|---|---|---|---|---|
| NVIDIA GB10 appliance | 128 GB | 44 to 95 t/s | 554 t/s | ~32 |
| RTX PRO 6000 Blackwell | 96 GB | 205 t/s | 2,790 t/s | ~128 |
| 1x H200 NVL | 143 GB | 202 t/s | 7,777 t/s | ~192 |
| 2x H200 NVLink | 287 GB | 211 t/s | 11,061 t/s | ~192+ |
| 4x H200 NVLink | 575 GB | 216 t/s | 14,071 t/s | ~192+ |
Two lessons for buyers. First: tensor-parallel scaling is sublinear for efficient MoE models (1x, then 1.42x, then 1.81x going from 1 to 2 to 4 cards): you add cards to fit bigger models (an 80B coder on 2-way, a 397B frontier-class model on 4-way) and to raise the ceiling, not to make one user faster. Second: the interconnect is everything. The identical 4-GPU setup with just one of the four links crossing PCIe instead of NVLink collapsed from 14,071 to 1,693 tok/s, an 8x penalty. This is why consumer platforms top out at one big GPU, workstation platforms (Threadripper PRO) at ~4, and why NVLink-bridged server platforms own the multi-GPU tier. The platform is as much the product as the GPU.
Start with one GB10. Add a second for a bigger model. Change tiers for more users.
The ladder above turns into a simple upgrade guide. One GB10 (128 GB) runs a 26 to 35B-class private model for a small team (roughly 32 comfortable concurrent streams), and disciplined agent loops raise output quality on the same hardware before you spend anything. A second GB10 pools memory to ~256 GB: it does not make any single user faster, but it unlocks models that simply don't fit one unit, including a 500K-token-context coder that scored within ~2 points of its cloud-hosted version on SWE-bench Verified. That's the honest pitch: the pair is a model-size and context play for compliance-bound teams, not a speed upgrade. Need many simultaneous users? Change tiers instead: one RTX PRO 6000 workstation serves ~5x a GB10 pair's aggregate, and a single H200 nearly 3x that again. Need a frontier-size model on-premises? That's the NVLink multi-H200 tier (287 to 575 GB pools), where the largest open models run entirely inside your compliance boundary.
For single-user work, a laptop iGPU now trades blows with a Grace-Blackwell box.
Gemma-4-26B-A4B (4B active) on an AMD Strix Halo laptop (Radeon 8060S iGPU, unified LPDDR5X) decodes 63 t/s single-stream on llama.cpp Vulkan (43 t/s via Ollama), clearly ahead of the NVIDIA GB10's best vLLM single-stream (44.4 t/s). It is model-dependent: Qwen3.6's MTP path keeps the GB10 ahead there (95 vs 65 t/s), and the same 96 GB unified pool loads gpt-oss-120b whole: a 120B-class MoE at 37 t/s on a laptop. The GB10 keeps the serving crown: vLLM batching scales it to 554 t/s at 32 concurrent streams, where llama.cpp cannot follow. For the Apple side of the single-user question, see Mac Studio M5 Ultra versus DGX Spark.
A near-cloud, CUI-capable coder already runs on our fleet.
On the official SWE-bench Verified harness (stratified-150, no LM judge), DeepSeek-V4-Flash local scored 67.3%, essentially tying cloud DeepSeek-V4-pro (69.6%), beating the fleet's Qwen3.6 (55.3%), about 15 points under Claude Opus-4.8 (82.0%). We own a private coder within reach of the frontier. The August re-run at the vendor's sampling settings, above, lifted the same model to 75.3%.
NVFP4 4-bit is lossless for some models, not all.
Unsloth's NVFP4 quants hold full quality for Gemma-4 (coding and research within noise of BF16) at half the VRAM on Blackwell. But Qwen3.6-35B develops a long-form runaway at NVFP4 that FP8 doesn't, so quant choice is per-model, not universal. The per-model detail is in Gemma-4 measured on four GPUs.
Serving config is a first-class correctness variable.
Gemma-4 with the wrong vLLM tool parser emitted zero tool calls: it looked agentically incapable when it was a one-line misconfiguration. With --tool-call-parser gemma4 it became the best tool-caller in the set. Both models then passed real end-to-end runs through our live agent gateway.
Never let a model grade its own family.
Self-judging inflates scores 20 to 37%. Our cross-judge gate forbids same-family grading, and headline studies now use a three-model frontier panel (median of GPT-5.4, Opus-4.8 and DeepSeek-V4; GLM-5.2 until August 2026), which repeatedly caught a single judge under-rating one contestant by ~0.3.
Have a workload you want scored the same way? Petronella Technology Group, Inc. runs candidate models against your own tasks on this fleet before you buy hardware. Request a workload evaluation
How to Use These LLM Benchmarks: A Buyer's Guide by Tier
Start with one GB10 for a small team, add a second GB10 when you need a bigger model or a longer context, and change tiers to an RTX PRO 6000 or an H200 pool when you need more simultaneous users. More GPUs never make a single user faster; they buy model size, context and concurrency.
Consumer buyer guides sort models by the VRAM of a gaming card. Our smallest tier is a 128 GB appliance, so every tier on this page runs models that would not fit those guides at all, and the trade-offs are different: memory bandwidth, interconnect and concurrency decide the outcome, not raw parameter count. Read the tiers in order; each one answers a different question.
One GB10 (128 GB). Run Qwen3.6-35B-A3B in NVFP4 with block-draft speculation (coding 0.977). The appliance is memory-bandwidth-bound, so mixture-of-experts models beat dense models by about 5.5x and speculation is the biggest lever you have. Expect 44 to 95 t/s for one user and 554 t/s peak aggregate at roughly 32 comfortable concurrent streams. It is also a prefill machine: 1,963 tok/s prefill against 8.3 t/s dense decode, a 237:1 ratio, which is why long-prompt, short-output work (RAG, summarization, classification, reranking) belongs here.
A GB10 pair (256 GB). The second unit does not make anyone faster. It pools memory so that models which do not fit one unit become possible, including DeepSeek-V4-Flash with block drafting (research 0.963, a 524K-token context, and 75.3% on SWE-bench Verified at the vendor's sampling settings, within about 2 points of its cloud-hosted version). Buy the pair for model size and context, not speed.
Four GB10 units on a 200G RoCE switch (512 GB). Only with a switch: cabled directly, four units decoded at 3.5 t/s against 8.3 for one unit. On the fabric, Qwen3.8-Flash-Next-NVFP4 delivers 40 to 77 t/s for one user and 167 to 184 t/s across six streams with a 2.6-million-token KV cache. If aggregate matters more than per-user speed, four independent single-unit endpoints reach about 345 t/s instead of 184.
One RTX PRO 6000 Blackwell (96 GB). The first tier built for many users. Qwen3.8-27B in NVFP4 with block-draft speculation scores 0.984 on coding, a single user sees 205 t/s, and the card serves 2,790 t/s peak at roughly 128 comfortable streams, about 5x the aggregate of a GB10 pair. On an identical model it carried 2,480 tok/s across 32 streams to the GB10's 554, about 4.5x.
One H200 NVL (143 GB). The same single-user speed class (202 t/s) but 7,777 t/s aggregate and about 192 comfortable users, nearly 3x the RTX PRO 6000. The step up buys concurrency and headroom, not a faster chat.
Two H200 NVLink (287 GB). Qwen3-Coder-Next-FP8 is the best coder in class on the hard set (0.939 at 2.6 seconds per case); the pool decodes 211 t/s for one user and 11,061 t/s aggregate. This is the tier where an 80B coder fits whole.
Four H200 NVLink (575 GB). GLM-5.3-Flash-FP8 (coding 0.987) and frontier-size open models such as Ornith-1.0-397B-FP8 (hard set 0.971) run entirely inside your compliance boundary at 216 t/s per user and 14,071 t/s aggregate. Two cautions from the sweep: tensor-parallel scaling is sublinear (1x, 1.42x and 1.81x going from 1 to 2 to 4 cards), and one PCIe link in place of NVLink collapsed the same setup from 14,071 to 1,693 tok/s. NVLink-bridged server platforms own this tier; workstation platforms top out at about 4 cards.
| Tier | Memory pool | Best-in-slot pick | What the tier buys |
|---|---|---|---|
| One GB10 | 128 GB | Qwen3.6-35B-A3B-NVFP4 + DSpark | A small team, prefill-heavy work, about 32 streams |
| GB10 pair | 256 GB | DeepSeek-V4-Flash-DSpark | Bigger models and a 524K-token context, not speed |
| Four GB10, 200G RoCE | 512 GB | Qwen3.8-Flash-Next-NVFP4 | A 125B pool with a 2.6-million-token KV cache |
| RTX PRO 6000 Blackwell | 96 GB | Qwen3.8-27B-NVFP4 + DSpark | Multi-user serving, about 128 streams |
| 2x H200 NVLink | 287 GB | Qwen3-Coder-Next-FP8 | An 80B coder whole, 11,061 t/s aggregate |
| 4x H200 NVLink | 575 GB | GLM-5.3-Flash-FP8 | Frontier-size open models, 14,071 t/s aggregate |
Which tier you can justify depends on the memory pool your target model needs, the number of concurrent users at peak and the data class you handle. Our on-premise AI server builds page covers the platforms behind each tier, and the engagement model below covers how we get from a benchmark row to a running deployment.
Engagement Model: From Benchmark Result to Private Deployment
Engagements are scoped after a 30-minute discovery call; cost depends on the hardware tier, the data class and whether Petronella Technology Group, Inc. operates the deployment for you.
The path from a row on this page to a running system has four steps. Discovery: which users, which tasks, which data class (CUI, PHI or privileged material) and what already exists on your network. Workload evaluation: we run candidate models against your own tasks on this fleet, under the same harness and judge rules described above, before you buy hardware, so the pick is measured on your work rather than inferred from ours. Build: the tier is specified from the picks above, the serving stack is configured with the flags recorded in the results (tool parsers, speculative decoding, sampling), and the deployment is stood up inside your compliance boundary. Operate: either your team runs it with our runbooks, or Petronella Technology Group operates it as part of its Private AI services, including model refreshes as new leaders appear on the leaderboards.
Nothing on this page is a quote. The discovery call is where the number comes from, and it is also where we tell you if a smaller tier, or no new hardware at all, would do the job.
Self-Hosted LLM Benchmarks FAQ
Short answers to the questions buyers ask most about these self-hosted LLM benchmarks; every figure below repeats a result shown above.
What do these self-hosted LLM benchmarks actually measure?
Can a model grade its own output?
What hardware were the scores measured on?
What is the best self-hosted model for coding right now?
How close is a local model to the cloud on SWE-bench Verified?
Does 4-bit NVFP4 quantization hurt quality?
Which model should run on a single NVIDIA GB10?
Does adding GPUs make a single user faster?
Why do serving settings matter as much as the model?
--tool-call-parser gemma4 it became the best tool-caller in the set. Mistral models need --tool-call-parser mistral. The draft-dspark flag on a plain GGUF is a silent no-op, so we verify the creating MTP draft context log line before believing any speculative-decoding number, and we publish the sampling configuration with every SWE-bench result.Which small model is safest for a voice agent that calls tools?
How often are these results updated?
Can Petronella Technology Group, Inc. run one of these models inside my compliance boundary?
About the Author
Craig Petronella is a CMMC Registered Practitioner (CMMC-RP), Cisco CCNA, CWNE and holder of Digital Forensic Examiner License 604180-DFE. He is the Amazon #1 best-selling author of 14+ cybersecurity books and the founding principal of Petronella Technology Group, Inc., which has served regulated and defense-adjacent organizations since 2002.
Petronella Technology Group, Inc. is a Cyber AB Registered Provider Organization (RPO-1449). Every score on this page was produced on hardware the company owns and operates in Raleigh, North Carolina, using the evaluation harness described in the methodology section; no vendor-supplied numbers are used.
Put the Results to Work
The benchmark tells you which model. These guides cover the rest: readiness, policy, compliance scope and how a private deployment is run day to day.
The Business AI Guide
Where private AI pays off, what it costs, and how to start without exposing data. Free download.
AI Readiness Checklist
The questions to answer before you buy hardware or sign an API agreement.
AI Acceptable Use Policy Template
A policy your staff can actually follow, written for regulated firms.
CMMC Compliance Hub
Why CUI workloads drive on-premises AI, and what an assessor expects to see.
AI Cybersecurity Solutions
Securing the models, the data pipeline and the hardware they run on.
Books by Craig Petronella
Practical references on CMMC, HIPAA and cybersecurity for business owners.
Talk to the team that measured every number on this page
A 30-minute call is enough to tell you which tier fits your users, your data and your budget, and what it will cost to run.
Petronella Technology Group, Inc. · 5540 Centerview Dr., Suite 200, Raleigh, NC 27606 · 919-348-4912 · info@petronellatech.com · Last Updated: August 29, 2026