← Full journal

LAB-01 · Local AI on my PC: Q4 vs Q8

A tool (llmfit) told me Qwen3-8B would fit on my PC. I wanted to check it and, while at it, see what changes between two versions of the same model: Q4, more compressed, and Q8, more precise and heavier. Do they load? How fast do they answer? Do they solve a hard problem better?

How I measured it. Hardware: RTX 5070 Ti, 16 GB. Models: Qwen3-8B in Q4_K_M and Q8_0. Tool: Ollama 0.40.16, on a local server.

  • Three rounds per version, with the same context (4096), the same three tasks and the same order. I saved all 18 answers unedited.
  • Temperature 0 and seed 42; I measured it with a script, and every figure is the median of the three rounds.
  • For the hard problem I wrote a new puzzle: 5 people, 5 days and 5 conditions. A program checked by brute force that it has exactly one solution.
  • I wrote it 5 different ways. Each model tried it without reasoning and with reasoning.

What loading showed. Both versions loaded and answered the three tasks in all three rounds, 100% on the GPU (37/37 layers).

Loading: median of three rounds
MeasureQ4_K_MQ8_0
Minimum estimated by llmfit (GB)5.819.66
VRAM allocated by Ollama (GB)5.598.90
Card memory, loaded minus idle (GB)5.819.10
Preload (s)2.152.40
Generation speed (tokens per second), median per task
TaskQ4_K_MQ8_0
Explain RAM114.6576.50
Order robots121.4979.00
Summarize a paragraph119.8979.51

The puzzle. Without reasoning, Q4 solved 1 of 5 wordings and Q8, 0 of 5. With reasoning, both solved 4 of 5; on the fifth they ran out of room and gave no answer. Without reasoning they took under 1 s; with reasoning, Q4 took 13 to 16.5 s and Q8 took 20 to 29.5 s.

Puzzle: wordings solved out of 5
ModeQ4_K_MQ8_0
Without reasoning1 of 50 of 5
With reasoning4 of 54 of 5

Q4 solving it without reasoning only happened with the original wording: with the others, both failed. In a chat-style repeat (factory settings, 3 rounds, 15 attempts per cell) the pattern held: without reasoning, Q4 solved 3 of 15 and Q8, 0 of 15; with reasoning, both solved 12 of 15.

Something visible at a glance: when ordering three robots, Q8 put Ciro at one end and broke a condition. And when asked for a two-sentence summary, Q4 gave four. Neither version came out clean.

What it does not prove. That llmfit is right on other machines, or that its memory figure is exact: only that these two setups loaded and answered here. It is not a quality score: it is one puzzle, five wordings, one machine and one setup. It does not say Q4 is for «everyday» and Q8 for «hard» work either: this test does not back that idea. A model fitting on the card does not mean it does your task well.

Watch the video for LAB-01