What can you run on a 24GB graphics card?
With 24GB VRAM you can run 13 of the 30 open-source models we track — up to about 32B parameters at Q4 quantization. The most popular that fits is Qwen3.5-9B, needing 6.8 GB and generating around ~115 tok/s on typical hardware of this size.
Typical machines: RTX 4090 · RTX 3090 · RX 7900 XTX. After the operating system takes its share, about 24 GB is free for a model.
Having the memory is not the same as having the speed
24GB VRAM decides whether a model loads. How fast it answers is decided by memory bandwidth — how quickly your machine can read the model, which it must do once for every word it writes. The table quotes a RTX 4090; pick yours below to see the difference. Why bandwidth and not the processor →
Models that run on 24GB VRAM in 2026
| Model | Size | Memory | Est. speed | Feels like |
|---|---|---|---|---|
| Qwen3.5-9BAlibaba / Qwen | 9.7B | 6.8 GBRuns well | ~115 tok/s | Fast |
| gpt-oss-20bOpenAI · MoE | 22B | 13 GBRuns well | ~282 tok/s4.3B active | Fast |
| Qwen3.5-4BAlibaba / Qwen | 4.7B | 3.8 GBRuns well | ~239 tok/s | Fast |
| Qwen3.5-0.8BAlibaba / Qwen | 873M | 1.3 GBRuns well | ~1230 tok/s | Fast |
| Qwen3.5-27BAlibaba / Qwen | 28B | 19 GBRuns well | ~39 tok/s | Fast |
| Qwen3.5-2BAlibaba / Qwen | 2.3B | 2.0 GBRuns well | ~512 tok/s | Fast |
| GLM-4.7-FlashZhipu AI | 31B | 22 GBTight | ~35 tok/s | Fast |
| Olmo-3-7B-InstructAi2 | 7.3B | 7.2 GBRuns well | ~147 tok/s | Fast |
| GLM-4.6V-FlashZhipu AI | 10B | 7.0 GBRuns well | ~106 tok/s | Fast |
| ERNIE-4.5-VL-28B-A3B-PTBaidu AI · MoE | 29B | 20 GBRuns well | ~37 tok/s | Fast |
| Olmo-3.1-32B-InstructAi2 | 32B | 23 GBTight | ~34 tok/s | Fast |
| ERNIE-4.5-21B-A3B-PTBaidu AI · MoE | 22B | 15 GBRuns well | ~49 tok/s | Fast |
| ERNIE-4.5-0.3B-PTBaidu AI | 361M | 853 MBRuns well | ~2723 tok/s | Fast |
All figures at Q4_K_M quantization, 4K context. “Memory” includes the model, its context cache and runtime overhead. Speed is calculated, not benchmarked: 1008 GB/s × 0.65 ÷ bytes read per token, assuming a RTX 4090. Scale it by your own machine's bandwidth from the list above. Mixture-of-experts models read only their active parameters per token, which is why some large ones outrun smaller dense models.
Image generators that also fit
These make pictures rather than text. Their memory use scales with the resolution you generate at, not conversation length, so treat these figures as a floor rather than a ceiling.
What you'd need more memory for
The next models up, and what they ask for.