What can you run on a 32GB graphics card?
With 32GB VRAM you can run 65 of the 101 open-source models we track — up to about 36B parameters at Q4 quantization. The most popular that fits is Qwen3.5-9B, needing 6.8 GB and generating around ~205 tok/s on typical hardware of this size.
Typical machines: RTX 5090. After the operating system takes its share, about 33 GB is free for a model.
Having the memory is not the same as having the speed
32GB VRAM decides whether a model loads. How fast it answers is decided by memory bandwidth — how quickly your machine can read the model, which it must do once for every word it writes. The table quotes a RTX 5090; pick yours below to see the difference. Why bandwidth and not the processor →
Models that run on 32GB VRAM in 2026
| Model↕ | Size↕ | Memory↑ | Est. speed↕ | Downloads↕ | Feels like |
|---|---|---|---|---|---|
| granite-embedding-97m-multilingual-r2Ibm | 97M | 655 MBRuns well | ~11044 tok/s | 237,787 | Fast |
| granite-embedding-311m-multilingual-r2Ibm | 312M | 748 MBRuns well | ~6190 tok/s | 47,167 | Fast |
| LFM2.5-Encoder-230MLiquid Ai | 230M | 793 MBRuns well | ~8399 tok/s | 13,450 | Fast |
| LFM2.5-230MLiquid Ai | 230M | 808 MBRuns well | ~7593 tok/s | 54,986 | Fast |
| ERNIE-4.5-0.3B-PTBaidu AI | 361M | 853 MBRuns well | ~4840 tok/s | 21,323 | Fast |
| nemotron-3.5-asr-streaming-0.6bNVIDIA | 638M | 1.1 GBRuns well | ~2349 tok/s | 1,052,774 | Fast |
| Qwen3.5-0.8BAlibaba / Qwen | 873M | 1.3 GBRuns well | ~2187 tok/s | 2,990,867 | Fast |
| LFM2.5-1.2B-InstructLiquid Ai | 1.2B | 1.4 GBRuns well | ~1594 tok/s | 555,431 | Fast |
| LFM2.5-VL-1.6BLiquid Ai | 1.6B | 1.4 GBRuns well | ~1594 tok/s | 78,655 | Fast |
| Nemotron-3-Embed-1B-BF16NVIDIA | 1.1B | 1.5 GBRuns well | ~1691 tok/s | 467,453 | Fast |
| cohere-transcribe-arabic-07-2026Cohere | 2.1B | 1.9 GBRuns well | ~934 tok/s | 51,389 | Fast |
| Hy-MT2-1.8BTencent | 2B | 1.9 GBRuns well | ~1028 tok/s | 49,645 | Fast |
| Qwen3.5-2BAlibaba / Qwen | 2.3B | 2.0 GBRuns well | ~909 tok/s | 2,468,876 | Fast |
| granite-speech-4.1-2bIbm | 2.3B | 2.4 GBRuns well | ~747 tok/s | 402,320 | Fast |
| LFM2.5-2.6BLiquid Ai | 2.7B | 2.5 GBRuns well | ~696 tok/s | 77,973 | Fast |
| LFM2.5-Audio-1.5BLiquid Ai | 1.5B | 2.6 GBRuns well | ~639 tok/s | 1,025 | Fast |
| Unlimited-OCRBaidu AI | 3.3B | 2.7 GBRuns well | ~597 tok/s | 2,836,694 | Fast |
| LocateAnything-3BNVIDIA | 3.8B | 2.8 GBRuns well | ~553 tok/s | 463,853 | Fast |
| granite-4.1-3bIbm | 3.4B | 3.0 GBRuns well | ~555 tok/s | 300,243 | Fast |
| Shieldstral-1.0-3BMistral AI | 3.8B | 3.1 GBRuns well | ~543 tok/s | 2,480 | Fast |
| Ministral-3-3B-Instruct-2512Mistral AI | 3.8B | 3.1 GBRuns well | ~543 tok/s | 544,419 | Fast |
| Nemotron-Labs-Diffusion-3B-BaseNVIDIA | 3.8B | 3.3 GBRuns well | ~504 tok/s | 35,110 | Fast |
| Cosmos3-EdgeNVIDIA | 3.9B | 3.3 GBRuns well | ~500 tok/s | 53,138 | Fast |
| Nemotron-3.5-Content-SafetyNVIDIA | 4.3B | 3.6 GBRuns well | ~468 tok/s | 176,924 | Fast |
| gemma-4-E2B-itGoogle DeepMind | 5.1B | 3.8 GBRuns well | ~375 tok/s | 4,002,947 | Fast |
| Qwen3.5-4BAlibaba / Qwen | 4.7B | 3.8 GBRuns well | ~425 tok/s | 6,199,203 | Fast |
| Fara1.5-4BMicrosoft | 4.5B | 4.0 GBRuns well | ~404 tok/s | 4,882 | Fast |
| NVIDIA-Nemotron-3-Nano-4B-BF16NVIDIA | 4B | 4.1 GBRuns well | ~411 tok/s | 712,701 | Fast |
| Fara-7BMicrosoft | 8.3B | 5.5 GBRuns well | ~249 tok/s | 2,192 | Fast |
| Hy-MT2-7BTencent | 8B | 5.7 GBRuns well | ~252 tok/s | 26,629 | Fast |
| LFM2.5-8B-A1BLiquid Ai · MoE | 8.5B | 5.9 GBRuns well | ~1913 tok/s1B active | 170,464 | Fast |
| Nemotron-3-Embed-8B-BF16NVIDIA | 8B | 5.9 GBRuns well | ~243 tok/s | 142,381 | Fast |
| gemma-4-E4B-itGoogle DeepMind | 8B | 6.2 GBRuns well | ~218 tok/s | 5,295,850 | Fast |
| Nemotron-Labs-Diffusion-8B-BaseNVIDIA | 8.5B | 6.2 GBRuns well | ~227 tok/s | 413,039 | Fast |
| Ministral-3-8B-Instruct-2512Mistral AI | 8.9B | 6.3 GBRuns well | ~224 tok/s | 119,379 | Fast |
| granite-4.1-8bIbm | 8.8B | 6.3 GBRuns well | ~228 tok/s | 4,213,980 | Fast |
| Ornith-1.0-9BDeepreinforce | 9B | 6.7 GBRuns well | ~207 tok/s | 2,250,422 | Fast |
| Qwen3.5-9BAlibaba / Qwen | 9.7B | 6.8 GBRuns well | ~205 tok/s | 11,767,971 | Fast |
| GLM-4.6V-FlashZhipu AI | 10B | 7.0 GBRuns well | ~189 tok/s | 282,562 | Fast |
| Fara1.5-9BMicrosoft | 9.4B | 7.0 GBRuns well | ~197 tok/s | 14,526 | Fast |
| Olmo-3-7B-InstructAi2 | 7.3B | 7.2 GBRuns well | ~260 tok/s | 406,776 | Fast |
| gemma-4-12B-itGoogle DeepMind | 12B | 9.4 GBRuns well | ~164 tok/s | 2,963,990 | Fast |
| Ministral-3-14B-Instruct-2512Mistral AI | 14B | 9.7 GBRuns well | ~141 tok/s | 198,169 | Fast |
| Nemotron-Labs-Diffusion-14BNVIDIA | 14B | 9.7 GBRuns well | ~141 tok/s | 6,554 | Fast |
| Cosmos3-NanoNVIDIA | 16B | 11 GBRuns well | ~122 tok/s | 313,164 | Fast |
| ERNIE-4.5-21B-A3B-PTBaidu AI · MoE | 22B | 15 GBRuns well | ~631 tok/s3B active | 44,971 | Fast |
| Voxtral-Small-24B-2507Mistral AI | 24B | 16 GBRuns well | ~81 tok/s | 262,513 | Fast |
| Magistral-Small-2506Mistral AI | 24B | 16 GBRuns well | ~81 tok/s | 49,182 | Fast |
| Qwen3.5-27BAlibaba / Qwen | 28B | 19 GBRuns well | ~70 tok/s | 2,637,888 | Fast |
| diffusiongemma-26B-A4B-itGoogle DeepMind · MoE | 26B | 19 GBRuns well | ~447 tok/s4B active | 1,952,156 | Fast |
| gemma-4-26B-A4B-itGoogle DeepMind · MoE | 27B | 20 GBRuns well | ~456 tok/s4B active | 10,976,594 | Fast |
| ERNIE-4.5-VL-28B-A3B-PTBaidu AI · MoE | 29B | 20 GBRuns well | ~643 tok/s3B active | 218,879 | Fast |
| Qwen3.6-27BAlibaba / Qwen | 28B | 20 GBRuns well | ~68 tok/s | 6,572,759 | Fast |
| granite-4.1-30bIbm | 29B | 20 GBRuns well | ~67 tok/s | 20,228 | Fast |
| Fara1.5-27BMicrosoft | 27B | 20 GBRuns well | ~66 tok/s | 3,208 | Fast |
| Hy-MT2-30B-A3BTencent · MoE | 30B | 20 GBRuns well | ~640 tok/s3B active | 13,753 | Fast |
| North-Mini-Code-1.0Cohere · MoE | 30B | 22 GBRuns well | ~303 tok/s6.1B active | 23,720 | Fast |
| diffusiongemma-26B-A4B-it-NVFP4NVIDIA · MoE | 14B | 22 GBRuns well | ~222 tok/s4B active | 1,508,724 | Fast |
| GLM-4.7-FlashZhipu AI | 31B | 22 GBRuns well | ~62 tok/s | 2,071,716 | Fast |
| Olmo-3.1-32B-InstructAi2 | 32B | 23 GBRuns well | ~60 tok/s | 51,153 | Fast |
| Ornith-1.0-35BDeepreinforce · MoE | 35B | 24 GBRuns well | ~275 tok/s7B active | 2,736,644 | Fast |
| gemma-4-31B-itGoogle DeepMind | 31B | 24 GBRuns well | ~64 tok/s | 11,028,273 | Fast |
| Qwen3.5-35B-A3BAlibaba / Qwen · MoE | 36B | 25 GBRuns well | ~634 tok/s3B active | 2,539,444 | Fast |
| Qwen3.6-35B-A3BAlibaba / Qwen · MoE | 36B | 25 GBRuns well | ~631 tok/s3B active | 5,939,238 | Fast |
| Qwen-AgentWorld-35B-A3BAlibaba / Qwen · MoE | 35B | 25 GBRuns well | ~608 tok/s3B active | 84,349 | Fast |
Sorted by memory, smallest first — click any column to change it.
All figures at Q4_K_M quantization, 4K context. “Memory” includes the model, its context cache and runtime overhead. Speed is calculated, not benchmarked: 1792 GB/s × 0.65 ÷ bytes read per token, assuming a RTX 5090. Scale it by your own machine's bandwidth from the list above. Mixture-of-experts models read only their active parameters per token, which is why some large ones outrun smaller dense models.
Image generators that also fit
These make pictures rather than text. Their memory use scales with the resolution you generate at, not conversation length, so treat these figures as a floor rather than a ceiling.
What you'd need more memory for
The next models up, and what they ask for.