Inkling-Small: hardware requirements
Inkling-Small needs about 166 GB of memory at Q4_K_M quantization: a 163 GB download plus room for context and a runtime allowance. That is beyond consumer hardware; use it through an API instead.
Will it run on your machine?
Use it without your own hardware
OpenRouter lists Inkling Small at $0.45 per million input tokens and $1.20 per million output tokens. Artificial Analysis measured hosted providers generating 220 tokens per second (Oct 29, 2026).
Price as listed on OpenRouter today. Speed is for hosted servers, not a home computer.
Download sizes by quantization
Lower quantization means a smaller file and less memory, at some cost to quality. Q4_K_M is the usual starting point. What is this?
| Quant | Download | RAM @ 4K | RAM @ 32K | Source |
|---|---|---|---|---|
| Q4_K_Mrecommended | 163 GB | 166 GB | 171 GB | unsloth/Inkling-Small-GGUF |
| Q5_K_M | 196 GB | 200 GB | 205 GB | unsloth/Inkling-Small-GGUF |
| Q6_K | 219 GB | 223 GB | 228 GB | unsloth/Inkling-Small-GGUF |
| Q8_0 | 280 GB | 284 GB | 289 GB | unsloth/Inkling-Small-GGUF |
| BF16 | 528 GB | 531 GB | 536 GB | unsloth/Inkling-Small-GGUF |
Download sizes are measured from the published files named in the Source column. The RAM columns add the context cache, calculated from the model's configuration, and a runtime allowance of 10% of the download (at least 0.5GB, at most 3GB).