Local models
2026-10-23

A 4B model against a 27B, on the same laptop

Is a small model that answers four times faster worth more than a big one that is right more often? 66 questions graded by code, then six real code edits graded by hidden tests.

NVIDIA released Nemotron 3.5 Bolt E4B on 2026-10-23. My agent model is Qwen 3.8 27B, compressed to 2 bits to fit the laptop's 16 GB graphics card. An earlier 28-question test couldn't separate them: both scored 26 or 27. So I built a harder one, with Codex and Grok writing the cases and harness to a spec, and every case checked before it ran. Like the rest of the local models, this is about the backup for when the cloud is down, not a replacement for frontier models.

The models

Nemotron 3.5 Bolt E4B new

NVIDIA's small assistant model, built on Gemma 4 E4B: 8 billion weights, of which about 4 billion do the work; the rest are per-layer lookup tables. Text only.

4-bit (Q4_K_M) · 6.2 GB · writes ~113 tok/s · 64k context

nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB

Qwen 3.8 27B GSQ-RCO IQ2_S agent

The model my coding agent runs. Stock Qwen 3.8 27B at 2 bits, with multi-token prediction.

2-bit (IQ2_S) · 9.6 GB · writes ~55–65 tok/s · 64k context

ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF

Both run with llama.cpp on a 16 GB RTX 5080 Laptop GPU, one at a time, at the sampling settings each model's makers recommend.

How it was tested

Part one, the model on its own. 66 questions written for this test, in seven kinds:

Math (12): word problems of six to ten steps, answered to the cent.
Logic (8): grid puzzles with two attributes, and truth-teller/liar puzzles.
Code (14): write a function to a strict spec, or fix a bug, graded by at least eight tests each.
Tools (8): chain two to five lookups (stock, prices, suppliers, a calculator), where a later lookup depends on an earlier one; in some, a tool fails once and has to be retried.
Long context (6): combine facts spread through 16,000 to 48,000 tokens of log, among look-alike distractors such as a superseded rate or another day's figure.
Instructions (10): exact line and word counts, a forbidden letter, a format inside a format.
Documents (8): answer from a 1,100–1,700-word document with amendments. Half the questions aren't answered by the document, and the right reply is NOT STATED.

No model grades another. Every answer key is computed by code: math by a reference solution, every logic puzzle by brute force over all possibilities, which must find exactly one answer, code by tests that a reference implementation must pass, tools by replaying the lookups. A case whose key and reference disagree doesn't run. Each question ran three times, with thinking on and with it off, so 198 samples per model per mode. Rates below come with 95% intervals (Wilson).

Part two, the model as a coding agent. Six small repositories, each with a requested change: unit handling in a cut list, a scheduler's signature, inventory reservations, input validation, a TSV exporter, a harvest report. The model drives the Pi coding agent, with no network, can read, edit and run the tests it can see, and hands back a patch. Hidden tests it never saw decide the grade: a task counts only if every visible and hidden test passes. Two runs per task, up to 20 minutes each.

On its own

	Qwen 27B thinking	Qwen 27B direct	Bolt thinking	Bolt direct
Math	35/36	28/36	27/36	30/36
Logic	24/24	19/24	21/24	21/24
Code	30/42	29/42	28/42	21/42
Tools	23/24	23/24	22/24	19/24
Long context	16/18	17/18	11/18	9/18
Instructions	28/30	7/30	14/30	7/30
Documents	24/24	24/24	24/24	22/24
All	91%	74%	74%	65%
	86–94%	68–80%	68–80%	58–71%
Seconds a question	55	18	14	7

Passes out of 3 samples × the questions in each kind. The second line under All is the 95% interval.

Faster but
less right?

That was the question I started with: is a model that is right 8 times in 10 but four times faster worth more than one that is right 9 times in 10?

At the same speed, the big model wins. Bolt thinking and Qwen not thinking take about the same time, 14 and 18 seconds, and score the same, 74%. Qwen thinking takes 55 seconds and is right 91% of the time.
Retries help only where something checks the answer. With thinking, Bolt got 56 of the 66 questions right at least once in three tries, but only 40 right all three times. When tests or a calculator catch the miss, a fast model can try again. When nothing checks it, as with a figure read from a document, the miss just stands.
Some questions it never gets. Bolt missed ten questions on every try even with thinking: four of the instruction questions, two long-context questions, at 32,000 and 48,000 tokens, the mower price below, and three code specs with real edge cases. Speed doesn't buy those back. Qwen thinking missed two on every try, both code, each time by running out of its 16,384-token budget.
As an agent, the speed disappears. In part two Bolt took 75 seconds a task and Qwen 84. Both made about 15 tool calls. Qwen's slower writing was cancelled out by being right, and Bolt's speed bought nothing.

As a coding agent

	Qwen 27B	Bolt
Cut-list units	2/2	0/2
Harvest report, empty beds	2/2	1/2
Scheduler signature	2/2	1/2
Inventory reservations	2/2	2/2
Weigh-in validation	0/2	0/2
Kit TSV exporter	2/2	0/2
Runs fully solved	10/12	4/12
	55–95%	14–61%
Hidden tests passed, average	98%	77%
Seconds a task	84	75

Thinking on for both. Bolt's misses were mostly close: seven of eight hidden tests on three runs, six of seven on another. The weigh-in task beat both models on the same test every time: it asks for any finite int to be accepted, and a 400-digit int is finite, but all four patches checked it with math.isfinite(), which raises an error on an int too big for a float. I kept the grade. It's a real bug the spec covers.

Three of
the questions

Math. Qwen thinking 3 of 3; Qwen direct, Bolt thinking and Bolt direct 0 of 3. Bolt went wrong at the May step.

A walk-behind mower has a list price of $1,860. The price changes in this order.

In March the price rises by 6 percent. In May the price falls by 4 percent of the March price, which is the price after the rise. In July a rebate of $90 is subtracted only when the price after the May change is strictly greater than $1,800. Otherwise the rebate is $0. In August a discount of 15 percent is taken only on the part of the price above $1,000. The first $1,000 is not discounted. A sales tax of 7.25 percent is then charged on the price after that discount.

What is the amount paid, in dollars? Do not round any intermediate value. Round only the final answer to 2 decimal places.

End your reply with a line of the form: ANSWER: <answer>

Instructions. Qwen thinking 3 of 3; every other combination 0 of 3. The misses wrote lines of 9 or 10 words, or an extra line.

Write shop steps, with all of these constraints at once.

1. The reply is exactly 5 lines.
2. Lines 1 through 4 begin, in order, with Step 1: and Step 2: and Step 3: and Step 4: . The word Step and the number-with-colon are two words.
3. Each of those four lines has exactly 8 words. A word is separated by a single space.
4. Line 3 contains the whole word clamp, in any case. The letter z does not appear anywhere in the reply.
5. Line 5 is exactly: ANSWER: clamp

The last line required above is the answer line. Do not write any line after it.

Long context, about 48,000 tokens of shop log. Qwen 2 of 3 in each mode; Bolt 0 of 3 in each. Bolt either said the 9 August hours weren't given, or used the 5 hours lost on 8 August instead of the 3 lost on 9 August.

The lower orchard stops outdoor work when that day's high is 95 F or higher. The lost-time cost is the number of outdoor hours the lower orchard crew lost that day, multiplied by the standing hourly rate. Use 9 August only. Ignore the upper orchard. Ignore a superseded rate card and a superseded morning sheet. If the high was under 95 F, the cost is 0. What is the lost-time cost in dollars?

The rest of the 66 stay private, so a future model can be tested on questions it can't have trained on.

Corrections
during the run

The instruction prompts contradicted themselves. Each one spelled out its exact last line, then the generic line every question carries told the model to end with an answer line again. Without thinking, both models obeyed both and wrote the answer line twice. I removed the generic line from those ten, made the case checker reject it from then on, and re-ran all ten for both models. The table above uses the re-run. It lifted Bolt direct from 3 to 7 of 30; the remaining misses are counting.
The agent's first runs on Qwen never reached the model. Pi sent a thinking level that Qwen's chat template doesn't accept, so every request failed in 15 seconds and was graded as an empty patch. The harness now records a server error as an infrastructure failure and retries it rather than counting it, and Pi maps that level to Qwen's own default. Qwen's twelve runs were then done again.

Limits

One laptop, one quantization of each model: Qwen at 2 bits, Bolt at 4. A less compressed Qwen would likely do better, and so might Bolt.
66 questions and 12 agent runs per model are small samples. The intervals are wide; read differences of a few points as ties.
One agent harness, Pi. Other harnesses may suit either model better. The plan is to test the same tasks under OpenCode and a few others next.
The questions are my own, in shop, farm and everyday settings. They measure what I need from a backup model, not general ability.

What I'll do

Qwen 27B stays the agent model. It solved 10 of 12 real edits to Bolt's 4, in about the same time.
Bolt doesn't get a job on the laptop. Its strength is speed on easy questions, and the laptop already has a fast small model for those.
This becomes the standing test. Every new local model gets the same 66 questions and the same six tasks, so the results line up.