Cohere Releases Command A+ Open Source Model with 25B Active Parameters, 128K Context
Cohere has released Command A+ as an open source model under Apache 2.0 license. The sparse mixture-of-experts architecture features 25 billion active parameters out of 218B total parameters, supports 128K input context length, and includes vision capabilities alongside tool use and reasoning features.
Command A+ at a glance
How Command A+ ranks now
| Benchmark | Score | Rank | Source |
|---|---|---|---|
| Arena Elo | 1354 Elo | 115 of 141 | LMArena, Nov 2026 ↗ |
| MMMU | 75.1% | 10 of 13 | llm-stats, Jul 2026 ↗ |
Rank is against every model in our database with a sourced score on that benchmark, each counted once with its best score.
Cohere Releases Command A+ Open Source Model with 25B Active Parameters, 128K Context
Cohere has released Command A+ (command-a-plus-05-2026) as an open source model under Apache 2.0 license. The sparse mixture-of-experts architecture features 25 billion active parameters out of 218B total parameters, supports 128K input context length with 64K output length, and includes multimodal vision capabilities.
Architecture and Specifications
Command A+ uses a decoder-only Sparse Mixture-of-Experts Transformer architecture with 128 experts, activating 8 per token plus one shared expert applied to all tokens. According to Cohere, the model employs a 3:1 ratio of sliding-window attention layers with Rotational Positional Embeddings to global attention layers without positional embeddings, a design first introduced in the earlier Command A model.
The sparse MoE layer is trained in a "fully dropless manner" using a token-choice router, with additive-bias-based load balancing to distribute token load across experts. The architecture replaces the standard softmax router activation function with a normalized sigmoid over the topk expert logits per token.
Deployment and Hardware Requirements
Cohere provides three quantization options with minimal quality differences:
- BF16 (16-bit): Requires 4x B200 or 8x H100 GPUs
- FP8 (8-bit): Requires 2x B200 or 4x H100 GPUs
- W4A4 (4-bit): Requires 1x B200 or 2x H100 GPUs
Cohere recommends the W4A4 quantization for most use cases, claiming "superior speed and latency characteristics alongside a smaller hardware footprint."
Capabilities
The model supports 48 languages including English, Chinese, Japanese, Arabic, Spanish, and various European and Asian languages. It includes native tool use capabilities trained for conversational API interactions, with support for JSON schema tool descriptions and citation generation to ground responses in specific tool results.
Command A+ includes a reasoning mode that generates explicit thinking steps between <START_THINKING> and <END_THINKING> tags before producing final outputs. The model also accepts image inputs for multimodal processing.
Integration
The model requires transformers installation from source and is compatible with vLLM 0.21.0 or higher. Tool calling and reasoning parsing require Cohere's melody library (version 0.9.0+). The model is available on Hugging Face with a hosted demo space for testing before deployment.
What This Means
Command A+ enters the competitive open source model space with a sparse MoE design similar to Mixtral and DeepSeek's architectures, but with significantly more total parameters (218B vs Mixtral 8x22B's 141B). The 128K context window matches GPT-4 Turbo and Claude 3 capabilities, while the Apache 2.0 license allows unrestricted commercial use. The model's combination of vision, reasoning, and tool use in a single open source package targets enterprise deployments that previously required closed-source API providers.
Related Articles
Tencent releases Hy4 flash: 220B MoE, 12B active parameters, 1M-token context
Tencent has released Hy4 flash, a multimodal Mixture-of-Experts model with 220B total parameters, 12B active per token, and a 1,048,576-token context window. Tencent calls it its most cost-effective Hy model to date and claims a per-task cost of 1/20 that of Hy4 on its internal WorkIndex benchmark.
KAIST MLP Lab releases KORMo-Web-11B: open LLM, VLM and web agent scoring 30.1 on Korean Online-Mind2Web
KAIST MLP Lab has released KORMo-Web-11B, an 11B-parameter bilingual model that combines text, vision and browser-agent capabilities in one set of weights under Apache-2.0. On 612 Online-Mind2Web tasks it scores 29.0 in English and 30.1 in Korean, according to the lab.
Tencent releases Hy4 Flash Base: 220B-parameter MoE, 12B active, 1M-token context
Tencent has published Hy4 Flash Base on Hugging Face: a 220B-parameter mixture-of-experts model with 12B active parameters per token, text and image input, and a 1M-token context. Tencent calls it its most cost-effective Hy model to date.
SenseNova-Loop-9B released: 9B parameters, 56 layer executions, 512K context
SenseNova has released SenseNova-Loop-9B, a roughly 9B-parameter multimodal model that runs 56 decoder-layer executions per forward pass from 32 distinct layers. It supports up to 512K tokens of context and, per the model card, needs about 15 GiB of GPU memory at 256K context with 4-bit weights and an 8-bit KV cache.
Comments
Loading...