What can you run on a 12GB graphics card?
With 12GB VRAM you can run 45 of the 101 open-source models we track — up to about 16B parameters at Q4 quantization. The most popular that fits is Qwen3.5-9B, needing 6.8 GB and generating around ~58 tok/s on typical hardware of this size.
Typical machines: RTX 4070 · RTX 5070 · RTX 3060 12GB · RX 7700 XT. After the operating system takes its share, about 12 GB is free for a model.
Having the memory is not the same as having the speed
12GB VRAM decides whether a model loads. How fast it answers is decided by memory bandwidth — how quickly your machine can read the model, which it must do once for every word it writes. Machines with 12GB VRAM span 360–672 GB/s, so the fastest is about 1.9× quicker than the slowest with identical capacity. The table quotes a RTX 4070; pick yours below to see the difference. Why bandwidth and not the processor →
Models that run on 12GB VRAM in 2026
| Model↕ | Size↕ | Memory↑ | Est. speed↕ | Downloads↕ | Feels like |
|---|---|---|---|---|---|
| granite-embedding-97m-multilingual-r2Ibm | 97M | 655 MBRuns well | ~3106 tok/s | 237,787 | Fast |
| granite-embedding-311m-multilingual-r2Ibm | 312M | 748 MBRuns well | ~1741 tok/s | 47,167 | Fast |
| LFM2.5-Encoder-230MLiquid Ai | 230M | 793 MBRuns well | ~2362 tok/s | 13,450 | Fast |
| LFM2.5-230MLiquid Ai | 230M | 808 MBRuns well | ~2136 tok/s | 54,986 | Fast |
| ERNIE-4.5-0.3B-PTBaidu AI | 361M | 853 MBRuns well | ~1361 tok/s | 21,323 | Fast |
| nemotron-3.5-asr-streaming-0.6bNVIDIA | 638M | 1.1 GBRuns well | ~661 tok/s | 1,052,774 | Fast |
| Qwen3.5-0.8BAlibaba / Qwen | 873M | 1.3 GBRuns well | ~615 tok/s | 2,990,867 | Fast |
| LFM2.5-1.2B-InstructLiquid Ai | 1.2B | 1.4 GBRuns well | ~448 tok/s | 555,431 | Fast |
| LFM2.5-VL-1.6BLiquid Ai | 1.6B | 1.4 GBRuns well | ~448 tok/s | 78,655 | Fast |
| Nemotron-3-Embed-1B-BF16NVIDIA | 1.1B | 1.5 GBRuns well | ~476 tok/s | 467,453 | Fast |
| cohere-transcribe-arabic-07-2026Cohere | 2.1B | 1.9 GBRuns well | ~263 tok/s | 51,389 | Fast |
| Hy-MT2-1.8BTencent | 2B | 1.9 GBRuns well | ~289 tok/s | 49,645 | Fast |
| Qwen3.5-2BAlibaba / Qwen | 2.3B | 2.0 GBRuns well | ~256 tok/s | 2,468,876 | Fast |
| granite-speech-4.1-2bIbm | 2.3B | 2.4 GBRuns well | ~210 tok/s | 402,320 | Fast |
| LFM2.5-2.6BLiquid Ai | 2.7B | 2.5 GBRuns well | ~196 tok/s | 77,973 | Fast |
| LFM2.5-Audio-1.5BLiquid Ai | 1.5B | 2.6 GBRuns well | ~180 tok/s | 1,025 | Fast |
| Unlimited-OCRBaidu AI | 3.3B | 2.7 GBRuns well | ~168 tok/s | 2,836,694 | Fast |
| LocateAnything-3BNVIDIA | 3.8B | 2.8 GBRuns well | ~155 tok/s | 463,853 | Fast |
| granite-4.1-3bIbm | 3.4B | 3.0 GBRuns well | ~156 tok/s | 300,243 | Fast |
| Shieldstral-1.0-3BMistral AI | 3.8B | 3.1 GBRuns well | ~153 tok/s | 2,480 | Fast |
| Ministral-3-3B-Instruct-2512Mistral AI | 3.8B | 3.1 GBRuns well | ~153 tok/s | 544,419 | Fast |
| Nemotron-Labs-Diffusion-3B-BaseNVIDIA | 3.8B | 3.3 GBRuns well | ~142 tok/s | 35,110 | Fast |
| Cosmos3-EdgeNVIDIA | 3.9B | 3.3 GBRuns well | ~141 tok/s | 53,138 | Fast |
| Nemotron-3.5-Content-SafetyNVIDIA | 4.3B | 3.6 GBRuns well | ~132 tok/s | 176,924 | Fast |
| gemma-4-E2B-itGoogle DeepMind | 5.1B | 3.8 GBRuns well | ~105 tok/s | 4,002,947 | Fast |
| Qwen3.5-4BAlibaba / Qwen | 4.7B | 3.8 GBRuns well | ~120 tok/s | 6,199,203 | Fast |
| Fara1.5-4BMicrosoft | 4.5B | 4.0 GBRuns well | ~114 tok/s | 4,882 | Fast |
| NVIDIA-Nemotron-3-Nano-4B-BF16NVIDIA | 4B | 4.1 GBRuns well | ~115 tok/s | 712,701 | Fast |
| Fara-7BMicrosoft | 8.3B | 5.5 GBRuns well | ~70 tok/s | 2,192 | Fast |
| Hy-MT2-7BTencent | 8B | 5.7 GBRuns well | ~71 tok/s | 26,629 | Fast |
| LFM2.5-8B-A1BLiquid Ai · MoE | 8.5B | 5.9 GBRuns well | ~538 tok/s1B active | 170,464 | Fast |
| Nemotron-3-Embed-8B-BF16NVIDIA | 8B | 5.9 GBRuns well | ~68 tok/s | 142,381 | Fast |
| gemma-4-E4B-itGoogle DeepMind | 8B | 6.2 GBRuns well | ~61 tok/s | 5,295,850 | Fast |
| Nemotron-Labs-Diffusion-8B-BaseNVIDIA | 8.5B | 6.2 GBRuns well | ~64 tok/s | 413,039 | Fast |
| Ministral-3-8B-Instruct-2512Mistral AI | 8.9B | 6.3 GBRuns well | ~63 tok/s | 119,379 | Fast |
| granite-4.1-8bIbm | 8.8B | 6.3 GBRuns well | ~64 tok/s | 4,213,980 | Fast |
| Ornith-1.0-9BDeepreinforce | 9B | 6.7 GBRuns well | ~58 tok/s | 2,250,422 | Fast |
| Qwen3.5-9BAlibaba / Qwen | 9.7B | 6.8 GBRuns well | ~58 tok/s | 11,767,971 | Fast |
| GLM-4.6V-FlashZhipu AI | 10B | 7.0 GBRuns well | ~53 tok/s | 282,562 | Fast |
| Fara1.5-9BMicrosoft | 9.4B | 7.0 GBRuns well | ~55 tok/s | 14,526 | Fast |
| Olmo-3-7B-InstructAi2 | 7.3B | 7.2 GBRuns well | ~73 tok/s | 406,776 | Fast |
| gemma-4-12B-itGoogle DeepMind | 12B | 9.4 GBRuns well | ~46 tok/s | 2,963,990 | Fast |
| Ministral-3-14B-Instruct-2512Mistral AI | 14B | 9.7 GBRuns well | ~40 tok/s | 198,169 | Fast |
| Nemotron-Labs-Diffusion-14BNVIDIA | 14B | 9.7 GBRuns well | ~40 tok/s | 6,554 | Fast |
| Cosmos3-NanoNVIDIA | 16B | 11 GBTight | ~34 tok/s | 313,164 | Fast |
Sorted by memory, smallest first — click any column to change it.
Step up to 16GB VRAM and 1 more model come within reach
Including ERNIE-4.5-21B-A3B-PT (22B) — around ~253 tok/s on a typical machine that size.
See everything 16GB VRAM runs →All figures at Q4_K_M quantization, 4K context. “Memory” includes the model, its context cache and runtime overhead. Speed is calculated, not benchmarked: 504 GB/s × 0.65 ÷ bytes read per token, assuming a RTX 4070. Scale it by your own machine's bandwidth from the list above. Mixture-of-experts models read only their active parameters per token, which is why some large ones outrun smaller dense models.
Image generators that also fit
These make pictures rather than text. Their memory use scales with the resolution you generate at, not conversation length, so treat these figures as a floor rather than a ceiling.
What you'd need more memory for
The next models up, and what they ask for.