What can you run on a 24GB graphics card?
With 24GB VRAM you can run 20 of the 51 open-source models we track — up to about 35B parameters at Q4 quantization. The most popular that fits is Qwen3.8-27B, needing 19 GB and generating around ~40 tok/s on typical hardware of this size.
Typical machines: RTX 4090 · RTX 3090 · RX 7900 XTX. After the operating system takes its share, about 24 GB is free for a model.
Having the memory is not the same as having the speed
24GB VRAM decides whether a model loads. How fast it answers is decided by memory bandwidth — how quickly your machine can read the model, which it must do once for every word it writes. The table quotes a RTX 4090; pick yours below to see the difference. Why bandwidth and not the processor →
Models that run on 24GB VRAM in 2026
| Model↕ | Size↕ | Memory↑ | Est. speed↕ | Downloads↕ | Feels like |
|---|---|---|---|---|---|
| LFM2.5-230MLiquid Ai | 230M | 808 MBRuns well | ~4271 tok/s | 58,011 | Fast |
| MiniCPM5-1BOpenBMB | 1.1B | 1.3 GBRuns well | ~1004 tok/s | 667,154 | Fast |
| MiniCPM5-2BOpenBMB | 2.5B | 2.3 GBRuns well | ~420 tok/s | 206,774 | Fast |
| LFM2.5-2.6BLiquid Ai | 2.7B | 2.5 GBRuns well | ~391 tok/s | 98,713 | Fast |
| LFM2.5-VL-3BLiquid Ai | 3.1B | 2.5 GBRuns well | ~391 tok/s | 31,296 | Fast |
| North-Micro-Vision-InstructCohere | 2.5B | 2.5 GBRuns well | ~437 tok/s | 120,354 | Fast |
| granite-4.2-3bIbm | 3.7B | 3.1 GBRuns well | ~292 tok/s | 32,017 | Fast |
| Mage-VLMicrosoft | 4.7B | 3.9 GBRuns well | ~241 tok/s | 29,690 | Fast |
| LFM2.5-8B-A1BLiquid Ai · MoE | 8.5B | 5.9 GBRuns well | ~1076 tok/s1B active | 80,632 | Fast |
| Ling-3.0-tinyInclusionai · MoE | 7.9B | 6.2 GBRuns well | ~679 tok/s1.6B active | 25,727 | Fast |
| granite-4.2-8bIbm | 8.8B | 6.6 GBRuns well | ~123 tok/s | 52,946 | Fast |
| Ornith-1.5-9BDeepreinforce | 9.7B | 6.9 GBRuns well | ~113 tok/s | 453,339 | Fast |
| gemma-4-12B-itGoogle DeepMind | 12B | 9.6 GBRuns well | ~91 tok/s | 2,958,440 | Fast |
| Qwen3.8-27BAlibaba / Qwen | 28B | 19 GBRuns well | ~40 tok/s | 7,703,400 | Fast |
| diffusiongemma-26B-A4B-itGoogle DeepMind · MoE | 26B | 19 GBRuns well | ~252 tok/s4B active | 708,393 | Fast |
| Muse-Glimmer-30BMeta AI | 30B | 20 GBRuns well | ~36 tok/s | 441,881 | Fast |
| granite-4.2-30bIbm | 29B | 21 GBTight | ~37 tok/s | 17,894 | Fast |
| North-Mini-Code-1.0Cohere · MoE | 30B | 22 GBTight | ~171 tok/s6.1B active | 11,742 | Fast |
| Laguna-XS-2.1Poolside · MoE | 33B | 23 GBTight | ~162 tok/s6.7B active | 32,445 | Fast |
| Nex-N2.5-miniNex Agi · MoE | 35B | 24 GBTight | ~155 tok/s7B active | 4,543 | Fast |
Sorted by memory, smallest first — click any column to change it.
Step up to 32GB VRAM and 2 more models come within reach
Including NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 (18B) and Ornith-1.5-35B-A3B (36B) — around ~275 tok/s on a typical machine that size.
See everything 32GB VRAM runs →All figures at Q4_K_M quantization, 4K context. “Memory” includes the model, its context cache and runtime overhead. Speed is calculated, not benchmarked: 1008 GB/s × 0.65 ÷ bytes read per token, assuming a RTX 4090. Scale it by your own machine's bandwidth from the list above. Mixture-of-experts models read only their active parameters per token, which is why some large ones outrun smaller dense models.
What you'd need more memory for
The next models up, and what they ask for.