← Benchmarks with RTX 3080

The chip, and what I want from it

Part 1 of 9 · Apr 19, 2026 · 5 min read

In short A 10 GB, 760 GB/s card should fit a quantized 8B model and generate fast on paper. Sharing the card and decision quality are the open risks.Full verdict ↓

I have one GPU: an RTX 3080 with 10 GB of memory, in the machine that runs my Kubernetes node. This series is the record of finding out what that single card can really do, written in the order it happened. It starts with the chip and the goal, because every later decision only makes sense against them.

10 GBGDDR6X on a 320-bit bus
760 GB/smemory bandwidth
29.8 TFLOPSFP32 peak, 8,704 CUDA cores
320 Wboard power, always on

The series at a glance

Post Question
2 What are the tests, and where can you get them?
3 Which engine runs it, and why did the first one not stay?
4 Which model fits?
5 How fast is it against the paper ceilings?
6 Which engine is fastest and cheapest to run?
7 Does a bigger model pay off, and did I reach the goal?

The chip on paper

RTX 3080 (10 GB)
Chip GA102, Ampere
Streaming multiprocessors 68
CUDA cores 8,704
Tensor cores 272 (3rd generation)
Boost clock about 1.71 GHz
Memory 10 GB GDDR6X, 320-bit, 19 Gbps
Board power 320 W

Two figures fall out of that sheet. Compute is 8,704 cores × 2 operations × 1.71 GHz, about 29.8 TFLOPS in FP32, with the tensor cores roughly doubling that for FP16. Memory bandwidth is a 40-byte-wide bus at 19 Gbps, which is 760 GB/s.

Divide one by the other and you get the card's balance: about 39 operations per byte moved. A job that does fewer operations on each byte it reads waits on memory and is bandwidth-bound. Generating text one token at a time is exactly that kind of job, so for language models the 760 GB/s matters more than the TFLOPS. The third number, 10 GB, is a cliff and not a slope: a model that needs 10.5 GB does not run slowly, it does not run.

The job I bought it for

My goal is fully local personal agents: software that works on my email, finances, health data and tasks, running on my own hardware. Three properties matter:

  • Private. The data never has to leave the house to be read by a model.
  • Always on. No cold starts, no quota, nothing that stops when a subscription lapses.
  • No per-token meter. An agent that loops a hundred times to finish a task should cost electricity, not dollars.

The cost is that all of it has to fit on one 10 GB card.

What I hoped it could carry

I did not start modest. The stack I tried to put on this one GPU, in the order I tried it:

Workload Why an agent stack needs it First tried Where it lives now
Chat and tool-calling LLM Every agent decision Apr GPU
Embeddings and rerankers Memory and retrieval Jun CPU
Real-time speech to text A voice interface Aug Removed
OCR Scanned documents Aug Removed
Speech synthesis, batch speech to text Voice in and out by Jun CPU

The story of the series is partly in that last column: the card began as the home of everything and ended as the home of one language model.

What the datasheet predicts

Before measuring anything, the specs already make claims. These are what the rest of the series tests:

Question What the paper says Tested in
Does a 7 to 8B model fit? 4-bit weights are 3.5 to 5 GB, which leaves room for context Post 4
How fast can it generate? At most bandwidth ÷ weight bytes: 217 tok/s for 3.5 GB, 151 for 5 GB Post 5
How fast can it read a prompt? Compute ÷ (2 × parameters), roughly 3,700 to 4,300 tok/s on FP16 numbers Post 5
Can several jobs share the card? Only if their memory sums to under 10 GB Post 3
Is a small model good enough to decide? The paper cannot say Post 7

What would count as done

Criterion How I measure it Instrument
It fits, stays resident, and can share VRAM actually used, GPU offload ollama ps, nvidia-smi
It is fast enough to feel interactive First-token time, tokens per second, throughput under load engine_bench.py, sweep.sh
It makes decisions I would trust Pass rate on tool-calling and routing cases llm_bench.py, the decision suite
One person can keep it running Number of moving parts, what breaks Notes in each post

The next post describes the instruments, and the rest follow the optimisation in order (the table at the top maps them).

Where it stands, and my verdict

Nothing is measured yet, so this is the starting position:

Criterion Status
Fits, resident, shareable paper says yes for one model
Fast enough paper gives ceilings, not results
Decisions I would trust unknown
Easy to run unknown

Verdict: worth testing. The datasheet says a small quantized model fits with room to spare and generates fast. The two things it cannot tell me, whether the card can be shared and whether a model this small decides well enough, are the ones that decide the goal, and they are what the next six posts are about.