Alibaba Qwen Releases 35B Sparse MoE Model with 262K Context and Multimodal Support

Alibaba Cloud has released Qwen3.6-35B-A3B, an open-weight sparse mixture-of-experts model with 35 billion total parameters but only 3 billion active parameters per token.

Architecture and Specifications

The model uses a hybrid sparse MoE architecture that combines Gated DeltaNet linear attention with standard gated attention layers, according to Alibaba. This design reduces computational requirements by activating only 3 billion parameters per token while maintaining the capacity of the full 35 billion parameter model.

Qwen3.6-35B-A3B supports a native context window of 262,144 tokens, extensible to 1 million tokens using YaRN (Yet another RoPE extensioN method). The model accepts text, image, and video inputs, making it a multimodal system.

Key Capabilities

The model includes:

• Reasoning mode: Integrated thinking capability with reasoning traces preserved across multi-turn conversations
• Function calling: Native support for tool use and function execution
• Structured output: Ability to generate formatted responses
• Multimodal processing: Handles text, images, and video inputs

Pricing and Availability

The model is available through OpenRouter at $0.1612 per million input tokens and $0.9653 per million output tokens. Alibaba has released it under the Apache 2.0 license, making the model weights freely available for commercial and research use.

The sparse MoE architecture positions Qwen3.6-35B-A3B as a cost-efficient alternative to dense models, as only 8.6% of parameters are active during inference.

What This Means

The sparse MoE approach with only 3B active parameters per token makes this 35B model competitive on inference cost with much smaller dense models while potentially retaining more knowledge capacity. The 262K native context window and multimodal capabilities make it suitable for document analysis and video understanding tasks. However, benchmark scores are not yet publicly available, making it difficult to assess performance relative to other models in its class. The Apache 2.0 license and availability through OpenRouter lower the barrier to adoption for developers seeking open-weight alternatives to proprietary models.