What can you run on a 16GB graphics card?
With 16GB VRAM you can run 46 of the 101 open-source models we track — up to about 22B parameters at Q4 quantization. The most popular that fits is Qwen3.5-9B, needing 6.8 GB and generating around ~82 tok/s on typical hardware of this size.
Typical machines: RTX 4080 · RTX 5080 · RTX 4060 Ti 16GB · RX 7800 XT · Arc A770. After the operating system takes its share, about 16 GB is free for a model.
Having the memory is not the same as having the speed
16GB VRAM decides whether a model loads. How fast it answers is decided by memory bandwidth — how quickly your machine can read the model, which it must do once for every word it writes. Machines with 16GB VRAM span 288–960 GB/s, so the fastest is about 3.3× quicker than the slowest with identical capacity. The table quotes a RTX 4080; pick yours below to see the difference. Why bandwidth and not the processor →
Models that run on 16GB VRAM in 2026
| Model↕ | Size↕ | Memory↑ | Est. speed↕ | Downloads↕ | Feels like |
|---|---|---|---|---|---|
| granite-embedding-97m-multilingual-r2Ibm | 97M | 655 MBRuns well | ~4419 tok/s | 237,787 | Fast |
| granite-embedding-311m-multilingual-r2Ibm | 312M | 748 MBRuns well | ~2477 tok/s | 47,167 | Fast |
| LFM2.5-Encoder-230MLiquid Ai | 230M | 793 MBRuns well | ~3361 tok/s | 13,450 | Fast |
| LFM2.5-230MLiquid Ai | 230M | 808 MBRuns well | ~3038 tok/s | 54,986 | Fast |
| ERNIE-4.5-0.3B-PTBaidu AI | 361M | 853 MBRuns well | ~1937 tok/s | 21,323 | Fast |
| nemotron-3.5-asr-streaming-0.6bNVIDIA | 638M | 1.1 GBRuns well | ~940 tok/s | 1,052,774 | Fast |
| Qwen3.5-0.8BAlibaba / Qwen | 873M | 1.3 GBRuns well | ~875 tok/s | 2,990,867 | Fast |
| LFM2.5-1.2B-InstructLiquid Ai | 1.2B | 1.4 GBRuns well | ~638 tok/s | 555,431 | Fast |
| LFM2.5-VL-1.6BLiquid Ai | 1.6B | 1.4 GBRuns well | ~638 tok/s | 78,655 | Fast |
| Nemotron-3-Embed-1B-BF16NVIDIA | 1.1B | 1.5 GBRuns well | ~677 tok/s | 467,453 | Fast |
| cohere-transcribe-arabic-07-2026Cohere | 2.1B | 1.9 GBRuns well | ~374 tok/s | 51,389 | Fast |
| Hy-MT2-1.8BTencent | 2B | 1.9 GBRuns well | ~411 tok/s | 49,645 | Fast |
| Qwen3.5-2BAlibaba / Qwen | 2.3B | 2.0 GBRuns well | ~364 tok/s | 2,468,876 | Fast |
| granite-speech-4.1-2bIbm | 2.3B | 2.4 GBRuns well | ~299 tok/s | 402,320 | Fast |
| LFM2.5-2.6BLiquid Ai | 2.7B | 2.5 GBRuns well | ~278 tok/s | 77,973 | Fast |
| LFM2.5-Audio-1.5BLiquid Ai | 1.5B | 2.6 GBRuns well | ~256 tok/s | 1,025 | Fast |
| Unlimited-OCRBaidu AI | 3.3B | 2.7 GBRuns well | ~239 tok/s | 2,836,694 | Fast |
| LocateAnything-3BNVIDIA | 3.8B | 2.8 GBRuns well | ~221 tok/s | 463,853 | Fast |
| granite-4.1-3bIbm | 3.4B | 3.0 GBRuns well | ~222 tok/s | 300,243 | Fast |
| Shieldstral-1.0-3BMistral AI | 3.8B | 3.1 GBRuns well | ~217 tok/s | 2,480 | Fast |
| Ministral-3-3B-Instruct-2512Mistral AI | 3.8B | 3.1 GBRuns well | ~217 tok/s | 544,419 | Fast |
| Nemotron-Labs-Diffusion-3B-BaseNVIDIA | 3.8B | 3.3 GBRuns well | ~201 tok/s | 35,110 | Fast |
| Cosmos3-EdgeNVIDIA | 3.9B | 3.3 GBRuns well | ~200 tok/s | 53,138 | Fast |
| Nemotron-3.5-Content-SafetyNVIDIA | 4.3B | 3.6 GBRuns well | ~187 tok/s | 176,924 | Fast |
| gemma-4-E2B-itGoogle DeepMind | 5.1B | 3.8 GBRuns well | ~150 tok/s | 4,002,947 | Fast |
| Qwen3.5-4BAlibaba / Qwen | 4.7B | 3.8 GBRuns well | ~170 tok/s | 6,199,203 | Fast |
| Fara1.5-4BMicrosoft | 4.5B | 4.0 GBRuns well | ~162 tok/s | 4,882 | Fast |
| NVIDIA-Nemotron-3-Nano-4B-BF16NVIDIA | 4B | 4.1 GBRuns well | ~164 tok/s | 712,701 | Fast |
| Fara-7BMicrosoft | 8.3B | 5.5 GBRuns well | ~100 tok/s | 2,192 | Fast |
| Hy-MT2-7BTencent | 8B | 5.7 GBRuns well | ~101 tok/s | 26,629 | Fast |
| LFM2.5-8B-A1BLiquid Ai · MoE | 8.5B | 5.9 GBRuns well | ~765 tok/s1B active | 170,464 | Fast |
| Nemotron-3-Embed-8B-BF16NVIDIA | 8B | 5.9 GBRuns well | ~97 tok/s | 142,381 | Fast |
| gemma-4-E4B-itGoogle DeepMind | 8B | 6.2 GBRuns well | ~87 tok/s | 5,295,850 | Fast |
| Nemotron-Labs-Diffusion-8B-BaseNVIDIA | 8.5B | 6.2 GBRuns well | ~91 tok/s | 413,039 | Fast |
| Ministral-3-8B-Instruct-2512Mistral AI | 8.9B | 6.3 GBRuns well | ~90 tok/s | 119,379 | Fast |
| granite-4.1-8bIbm | 8.8B | 6.3 GBRuns well | ~91 tok/s | 4,213,980 | Fast |
| Ornith-1.0-9BDeepreinforce | 9B | 6.7 GBRuns well | ~83 tok/s | 2,250,422 | Fast |
| Qwen3.5-9BAlibaba / Qwen | 9.7B | 6.8 GBRuns well | ~82 tok/s | 11,767,971 | Fast |
| GLM-4.6V-FlashZhipu AI | 10B | 7.0 GBRuns well | ~76 tok/s | 282,562 | Fast |
| Fara1.5-9BMicrosoft | 9.4B | 7.0 GBRuns well | ~79 tok/s | 14,526 | Fast |
| Olmo-3-7B-InstructAi2 | 7.3B | 7.2 GBRuns well | ~104 tok/s | 406,776 | Fast |
| gemma-4-12B-itGoogle DeepMind | 12B | 9.4 GBRuns well | ~65 tok/s | 2,963,990 | Fast |
| Ministral-3-14B-Instruct-2512Mistral AI | 14B | 9.7 GBRuns well | ~57 tok/s | 198,169 | Fast |
| Nemotron-Labs-Diffusion-14BNVIDIA | 14B | 9.7 GBRuns well | ~57 tok/s | 6,554 | Fast |
| Cosmos3-NanoNVIDIA | 16B | 11 GBRuns well | ~49 tok/s | 313,164 | Fast |
| ERNIE-4.5-21B-A3B-PTBaidu AI · MoE | 22B | 15 GBTight | ~253 tok/s3B active | 44,971 | Fast |
Sorted by memory, smallest first — click any column to change it.
Step up to 24GB VRAM and 15 more models come within reach
Including gemma-4-26B-A4B-it (27B), Qwen3.6-27B (28B) and Ornith-1.0-35B (35B) — around ~257 tok/s on a typical machine that size.
See everything 24GB VRAM runs →All figures at Q4_K_M quantization, 4K context. “Memory” includes the model, its context cache and runtime overhead. Speed is calculated, not benchmarked: 717 GB/s × 0.65 ÷ bytes read per token, assuming a RTX 4080. Scale it by your own machine's bandwidth from the list above. Mixture-of-experts models read only their active parameters per token, which is why some large ones outrun smaller dense models.
Image generators that also fit
These make pictures rather than text. Their memory use scales with the resolution you generate at, not conversation length, so treat these figures as a floor rather than a ceiling.
What you'd need more memory for
The next models up, and what they ask for.