Alibaba Qwen Releases 27B Parameter Model with 262K Context Window, Claims 77.2% on SWE-bench Verified
Alibaba Qwen released Qwen3.6-27B, a 27-billion parameter model with a 262,144 token context window extensible to 1,010,000 tokens. The model claims 77.2% on SWE-bench Verified and 53.5% on SWE-bench Pro, with open weights available on Hugging Face.
Qwen3.6 27B — Quick Specs
Qwen3.6-27B Released with 262K Context Window
Alibaba Qwen released Qwen3.6-27B, a 27-billion parameter open-weight language model with a 262,144 token context window natively, extensible to 1,010,000 tokens. The model represents the first release in the Qwen3.6 series following the February 2025 Qwen3.5 launch.
Architecture Details
Qwen3.6-27B uses a non-standard transformer architecture with 64 layers and a 5,120 hidden dimension. The model employs a hybrid attention mechanism: 16 blocks of 3 "Gated DeltaNet" layers followed by 1 "Gated Attention" layer per block.
The Gated DeltaNet uses 48 linear attention heads for V and 16 for QK with 128 head dimension. The Gated Attention uses 24 attention heads for Q and 4 for KV with 256 head dimension. The model has 248,320 tokens in its vocabulary (padded) and uses rotary position embeddings with 64 dimensions.
Benchmark Performance
According to Alibaba, Qwen3.6-27B achieves:
- SWE-bench Verified: 77.2% (vs. 80.9% for Claude 4.5 Opus)
- SWE-bench Pro: 53.5% (vs. 57.1% for Claude 4.5 Opus)
- SWE-bench Multilingual: 71.3%
- Terminal-Bench 2.0: 59.3% (tied with Claude 4.5 Opus)
- SkillsBench Avg5: 48.2%
- MMLU-Pro: 86.2%
- GPQA Diamond: 87.8%
- AIME 2026: 94.1%
The company evaluated models using internal agent scaffolds with temperature 1.0, top_p 0.95, and 200K context windows for SWE-bench series tests. All benchmarks used 256K context windows unless specified otherwise.
Vision Capabilities
The model includes a vision encoder for multimodal tasks. According to Alibaba's benchmarks:
- MMMU: 82.9%
- MathVista mini: 87.4%
- VideoMME (with subtitles): 87.7%
- AndroidWorld: 70.3%
- Visual Agent V*: 94.7%
Key Features
Qwen3.6-27B introduces "thinking preservation" to retain reasoning context from historical messages during iterative development. The model supports multi-token prediction (MTP) during training and can be deployed with speculative decoding for faster inference.
The model is compatible with Hugging Face Transformers, vLLM (version 0.19.0+), SGLang (version 0.5.10+), and KTransformers. Alibaba recommends maintaining at least 128K token context length to preserve reasoning capabilities, though the default is 262K tokens.
Availability
Pricing has not been disclosed. Open weights are available on Hugging Face under the repository Qwen/Qwen3.6-27B. The model can be served via OpenAI-compatible APIs using standard inference frameworks.
What This Means
Qwen3.6-27B enters the competitive 20-30B parameter space with strong coding performance claims, particularly on repository-level tasks. The 262K native context window and 1M token extensibility position it for long-context applications, though real-world performance at extended lengths requires independent verification. The hybrid Gated DeltaNet/Attention architecture is unconventional and may offer efficiency advantages, but deployment complexity compared to standard transformers remains to be seen in production environments.
Related Articles
Nvidia releases Nemotron 3.5 Super VL: open 120B multimodal MoE with 12B active parameters
Nvidia has released Nemotron 3.5 Super VL, an open multimodal Mixture-of-Experts model with 120 billion total and 12 billion active parameters, built for continuously running enterprise agents. Two compact Bolt models for local PCs and edge devices launch alongside it, though the 4 GB variant ships later this month.
Unbiased releases Pareto 26.9: 262K-context composite model at $2.50/$7.50 per 1M tokens
Unbiased has released Pareto 26.9, a multimodal composite model with a 262K-token context window, priced at $2.50 per 1M input tokens and $7.50 per 1M output tokens. It is listed on OpenRouter with a Oct 13, 2026 release date. A cheaper 1.0M-context preview, Pareto 26.10, is already listed alongside it.
NVIDIA releases Nemotron 3.5 Super VL NVFP4: 120B-parameter, 12B-active vision-language model with 1M context
NVIDIA has published Nemotron 3.5 Super VL in NVFP4 quantization on Hugging Face: a 120B-parameter, 12B-active hybrid Mamba-2/Transformer MoE with a 1M-token context window and text, image and video input. NVIDIA reports the quantized checkpoint stays close to BF16 on most benchmarks. It fits on one DGX Spark, one H200, two H100s or one GB200.
DeepSeek V4.1 Flash 1017 launches with 552B CED backbone, 1M context, $0.15/$0.60 per 1M tokens
DeepSeek has released V4.1 Flash 1017, an October 17, 2026 snapshot built on its new Causal Encoder-Decoder architecture. It uses a 552B-parameter sparse MoE backbone with a 1M-token context window, native image understanding, and pricing of $0.15 input / $0.60 output per 1M tokens on OpenRouter.
Comments
Loading...