Liquid AI publishes LFM3-1.7B GGUF with QAD Q4_0 build at 1.05 GB for local CPU and GPU inference

Liquid AI has published LFM3-1.7B-GGUF on Hugging Face, a GGUF version of LFM3-1.7B built for local inference on CPUs and GPUs. The repository's Q4_0 file is a Quantization-Aware Distillation (QAD) checkpoint, LFM3-1.7B-QAD-Q4_0.gguf, which Liquid AI says was trained to recover the accuracy lost to Q4_0 quantization.

Architecture and usage details are in the original model card. The repository lists the architecture as lfm3 and the model size as 2B params. Hugging Face reports 97,184 downloads for the last month (as of October 2026).

Quantization options and file sizes

Precision	Format	Size
4-bit	Q4_0 (QAD)	1.05 GB
4-bit	Q4_K_M	1.1 GB
5-bit	Q5_K_M	1.26 GB
6-bit	Q6_K	1.43 GB
8-bit	Q8_0	1.85 GB
16-bit	BF16	3.47 GB
16-bit	F16	3.47 GB

Runtime support

Liquid AI lists three ways to run the model:

• Llama.app (macOS and Windows): requires llama.cpp version v0.8.0 or newer.
• LM Studio (macOS, Windows and Linux): requires LM Studio with llama.cpp engine 2.65.1 or newer.
• Ollama (ollama run lfm3, macOS): requires Ollama version v0.40.12 or newer.

llama.cpp support begins at b11960 (v0.8.0). Users pick a file by adding a quantization tag, for example llama-cli -hf LiquidAI/LFM3-1.7B-GGUF:Q4_K_M. To run the QAD build, point llama.cpp at the file directly with --hf-file LFM3-1.7B-QAD-Q4_0.gguf.

Recommended sampling

• General use: temperature 0.1, top_k 50, repeat penalty 1.1.
• Math and science: temperature 1.0, top_p 0.95, top_k 20.

Lineage and availability

The model tree lists LiquidAI/LFM3-1.7B-Base as the base model and LiquidAI/LFM3-1.7B as the finetuned model. The repository shows 62 quantized derivatives of this model and 6 Spaces using it. No Inference Provider currently deploys it, so it is aimed at local use rather than hosted API access. Pricing is not applicable to the local files, and a context window figure is not listed on this page.

What this means

The repository gives local users a range of file sizes from 1.05 GB to 3.47 GB, with one 4-bit option that Liquid AI has specifically trained to limit quantization loss. That QAD Q4_0 build is the notable detail: it is a distinct checkpoint rather than a standard post-training quantization. Anyone running it should check their runtime against the stated minimum versions (llama.cpp v0.8.0, LM Studio engine 2.65.1, Ollama v0.40.12), since older builds are not listed as supported. The separate sampling settings for math and science mean default configurations may need adjusting by task.