Nov · 21 · 2026 · permalink
R-01 · A follower challenged me: does a bigger AI get it right more often?
A follower told me: «try a bigger AI». So I compared the 8B in Q4, the 8B in Q8 and a 14B in Q4, with the same puzzle from LAB-01. Does a bigger AI get it right more often?
How I measured it. Hardware: RTX 5070 Ti, 16 GB. Models: 8B Q4, 8B Q8 and 14B Q4. Tool: Ollama 0.40.16, on the same local server.
- Context 4096. For speed, three rounds per model, rotating positions and with the runtime empty before each round.
- The LAB-01 puzzle (5 wordings) in two modes: deterministic (temperature 0, seed 42) and chat-style (factory settings, seeds 1, 2 and 3, 15 attempts per model and mode).
- Each model tried it without reasoning and with reasoning (up to 1024 output tokens direct and 3500 with reasoning).
- I measured it with a script and a checker that compares against the single solution, enumerated by brute force.
| Measure | 8B Q4 | 8B Q8 | 14B Q4 |
|---|---|---|---|
| GGUF weights (bytes) | 5 225 374 496 | 8 851 075 872 | 9 276 184 896 |
| Card memory increase (GB) | 5.805 | 9.138 | 9.880 |
| Layers on GPU | 37/37 | 37/37 | 41/41 |
| Generation, explain RAM (tokens/s) | 124.08 | 80.93 | 72.81 |
| Generation, order robots (tokens/s) | 126.88 | 85.05 | 77.06 |
| Generation, summarize (tokens/s) | 130.62 | 84.92 | 77.08 |
The 14B generated 9.2% to 10.0% fewer tokens per second than the 8B Q8 and 39.3% to 41.3% fewer than the 8B Q4. All three fit entirely on the card.
| Mode | 8B Q4 | 8B Q8 | 14B Q4 |
|---|---|---|---|
| Direct | 3 of 15 | 0 of 15 | 0 of 15 |
| With reasoning | 12 of 15 | 12 of 15 | 13 of 15 |
What I saw. The 14B got one more attempt right than each 8B when reasoning, and the difference came from a single wording (v4) with one seed. In deterministic mode, all three solved the same 4 of 5 wordings when reasoning. What changed the result most was letting it reason, not size. In exchange, each attempt took longer: on the original wording, with reasoning, 16.20 s on the 8B Q4, 24.72 s on the 8B Q8 and 26.00 s on the 14B.
I also gave the hardest wording more room (context 8192 and up to 7000 output tokens). All three solved 1 of 3 in chat; in deterministic mode none solved it.
What it does not prove. That big models are not worth it: it is one puzzle, five wordings, one machine and one setup. The three seeds repeat the same questions, and in the extended test I changed the context and the maximum output at the same time, so I do not know which one mattered. One more hit out of 15 is a small difference to build a rule on.
Next challenge. Getting 15 out of 15. No model managed it. I will try in a future lab, and if you have an idea for getting there, tell me.