StepFun Releases Step-3.7-Flash: 198B-Parameter Sparse MoE Model With 256K Context in GGUF Format
StepFun has released Step-3.7-Flash, a 198B-parameter sparse Mixture-of-Experts vision-language model that activates approximately 11B parameters per token. The model supports a 256K context window, native image understanding via a 1.8B-parameter vision encoder, and offers three selectable reasoning levels.
Step-3.7-Flash — Quick Specs
StepFun Releases Step-3.7-Flash: 198B-Parameter Sparse MoE Model With 256K Context in GGUF Format
StepFun has released Step-3.7-Flash in GGUF quantization format, a 198B-parameter sparse Mixture-of-Experts vision-language model that activates approximately 11B parameters per token. The model is now available for local deployment on consumer hardware with 128GB of unified memory.
Model Architecture and Capabilities
Step-3.7-Flash combines a 196B-parameter language backbone with a 1.8B-parameter vision encoder for native image understanding. According to StepFun, the sparse architecture enables throughput up to 400 tokens per second while maintaining a 256K token context window.
The model offers three selectable reasoning levels—low, medium, and high—designed to balance speed, cost, and reasoning depth. StepFun positions the model for agentic workloads including tool calling, multi-step reasoning, code generation, and mathematics, with native multilingual support.
Quantization Options and Memory Requirements
StepFun released seven quantization variants ranging from BF16 (394GB) to IQ3_XXS (76GB). The recommended Q4_K_S quantization requires 112GB, enabling full 256K context inference on devices with 128GB unified memory, including Apple Mac Studio (M4 Max), NVIDIA DGX Spark (GB10), and AMD Ryzen AI Max+ 395.
All quantizations below Q8_0 use imatrix calibration. A separate 4GB vision projector (mmproj) file enables multimodal inference when paired with any language quantization.
Benchmark Performance
On Apple Mac Studio (M4 Max, 128GB), the Q4_K_S quantization achieved:
- 420 tokens/second prompt processing at 2K context
- 48.5 tokens/second generation at 2K context
- 110 tokens/second prompt processing at 262K context
- 9.7 tokens/second generation at 262K context
On NVIDIA DGX Spark (GB10, 128GB), the same quantization reached 753 tokens/second prompt processing at 8K context and 26 tokens/second generation at 2K context.
The IQ4_XS quantization (105GB) delivered comparable performance with 7GB less memory usage.
Implementation Details
The model requires a custom branch of llama.cpp maintained by StepFun. Users must build from the step3.7 branch to access compatibility with the model's architecture. StepFun provides command-line inference tools and an OpenAI-compatible server for both text-only and multimodal workloads.
Pricing information has not been disclosed. The model files are available on Hugging Face.
What This Means
Step-3.7-Flash represents a significant accessibility milestone for large-scale vision-language models. By activating only 11B of 198B parameters per token, StepFun has made a model with GPT-4-class parameter count deployable on high-end consumer hardware. The 256K context window—previously limited to cloud-based models—can now run locally at speeds viable for production use cases. However, generation speeds drop substantially at maximum context (under 10 t/s at 262K), suggesting practical use will favor shorter contexts. The model's viability for production depends on benchmark scores against established models, which StepFun has not yet published.
Related Articles
GLM-5.3-Flash Debuts as Zhipu AI's First Natively Multimodal Model, 320B Parameters with 18B Active
Zhipu AI has released GLM-5.3-Flash, the first natively multimodal model in its GLM-5 series, built on a 320B-parameter mixture-of-experts architecture with only 18B active parameters. The company claims it outperforms GLM-5.2 while approaching Claude Opus 4.8 on coding and agentic benchmarks at a fraction of the cost. Unsloth has published quantized GGUF versions for local inference.
Qwen3.8-Flash-Next Debuts with 125B-Parameter Hybrid Architecture, Previews Qwen4 Design
Qwen3.8-Flash-Next is an experimental preview of the architecture Alibaba's Qwen team plans to use for Qwen4, combining hybrid attention, gated residuals, and n-gram embeddings in a 125B-parameter model with only 6B activated per token. Unsloth has released Dynamic 3.0 GGUF quantizations for local inference.
Tencent Open-Sources Hy4 Preview: 770B-Parameter MoE Model with 1M-Token Context
Tencent's Hy Team has open-sourced Hy4 preview, a 770-billion-parameter Mixture-of-Experts model with 49 billion activated parameters and a 1-million-token context window. The model is available under Apache 2.0 alongside an FP8-quantized variant, with Tencent claiming it beats GLM 5.3 and Kimi K3 on internal engineering evaluations.
Z.ai's GLM-5.3-Flash Matches Top Models at 7.5x Lower Cost, Runs Entirely on Chinese Chips
Z.ai released GLM-5.3-Flash, a 320-billion-parameter MoE model with an 18-billion active parameter count and a one-million-token context window. It nearly matches the larger GLM-5.3 on Artificial Analysis's Intelligence Index while costing roughly 7.5 times less per task, and it reportedly runs entirely on Chinese AI chips instead of Nvidia GPUs.
Comments
Loading...