Unsloth

NVIDIA released Nemotron-3.5-Bolt-E4B-4GB on 2026-10-23, a small language model post-trained from Gemma-4-E4B-it with 8B total (4B effective) parameters and 64K-token input and output. It targets local voice and device assistants. Unsloth now lists a GGUF build of the model on Hugging Face.

Unsloth has released GGUF quantizations of EmbeddingGemma 2, Google DeepMind's open 740M-parameter embedding model for text, images, video and audio. The base model maps all four modalities into one 768-dimensional space with an 8K context window under Apache 2.0.

Unsloth has published GGUF quantizations of Muse Glimmer-30B, a dense 29.6B-parameter causal transformer with a dedicated perception encoder, attributed to Meta Superintelligence Lab in the model card. The model targets autonomous agentic tasks on consumer hardware with a 131,072-token context window and 4-bit quantization under 20GB.

Unsloth has released GGUF quantizations of Kimi K3, a 2.8-trillion-parameter open-weight Mixture-of-Experts model from Moonshot AI with a 1-million-token context window and native vision support. The largest lossless quantization (Q8) weighs in at 1.56TB.

Unsloth has released optimized GGUF quantizations of DeepSeek-V4-Flash, a 284B parameter Mixture-of-Experts model that activates 13B parameters and supports 1 million token context windows. The Q8 quantization (UD-Q8_K_XL) runs at 162GB with claimed lossless precision, only 7GB larger than the Q4 variant.

IBM has released the Granite 4.1 family of language models under Apache 2.0 license. The models come in 3B, 8B, and 30B parameter sizes. Unsloth has released 21 GGUF quantized variants of the 3B model ranging from 1.2GB to 6.34GB.