← All writing

Benchmarks with RTX 3080

What I wanted from a 10 GB RTX 3080, the tests I wrote to find out, and how I tuned engines and models until it fit. Test code and cases are published.

  1. 1The chip, and what I want from itA 10 GB, 760 GB/s card should fit a quantized 8B model and generate fast on paper. Sharing the card and decision quality are the open risks.Apr 19, 2026
  2. 2The tests I wroteSpeed and fit can be repeated by anyone from the published folder. Quality rests on suites I wrote for my own agent; the journey suite is the current measure.Nov 9, 2026
  3. 3vLLM first, and why it did not stayvLLM claimed 8.5 of 10 GiB for a 7B model and sharing failed twice, so Ollama became the single GPU tenant.Apr 19, 2026
  4. 4Fitting a real model in 10 GBQwen3 8B at 40k context with a q8_0 KV cache: the only candidate that is fully resident and best at choosing tools.Sep 26, 2026
  5. 5Raw speed against the ceilingsThe 8B generates at 79% of the bandwidth ceiling and reads prompts faster than my ceiling allowed, so the ceiling was wrong. The 35B runs but is 1.8x slower to generate.Nov 11, 2026
  6. 6Engines head to headStay on Ollama: it costs 2 to 8% decode speed. In my run Strata did not reuse a prompt, which made a warm start 42x slower.Nov 11, 2026
  7. 7Does a bigger model pay off?Stay on the 35B: on the corrected journey suite it passes 72 of 81 and the 8B 32 of 81. The earlier suite's 9 of 24 ceiling was a measurement fault, and the analysis below is kept as its record.Nov 11, 2026
  8. 8The 35B with parallel slots and image inputParallel slots raise the 35B's throughput from 47 to 71 tok/s at four requests, and image input costs nothing in text speed. It still cannot reuse a prompt and still needs the rest of the stack switched off — but on the corrected journey suite it passes 72 of 81, against the 8B's 32.Nov 16, 2026
  9. 9A faster challenger: Gyro-SGyro-S decodes 1.5 to 1.7 times faster than Ollama's 8B, restarts a known prompt in 0.065 s, and passes 71 of 81 journeys on the suite prompt (the 35B 72, the 8B 32). On the production prompt the 35B fast holds 72 and Gyro-S falls to 62, so the 35B fast is still the safer decision-maker. The speed is the best in the series; the open question is fit.Nov 16, 2026