model releaseCohere

Cohere Releases Command A+: 218B-Parameter MoE Model With 4-Bit Quantization Runs on Single B200 GPU

TL;DR

Cohere has released Command A+, an open-source sparse mixture-of-experts model with 218 billion total parameters and 25 billion active parameters. The model features W4A4 quantization allowing deployment on a single Nvidia B200 GPU, supports 128K input context, and includes built-in chain-of-thought reasoning with vision capabilities.

2 min read
0

Command A+ at a glance

Context window192K tokens
Input$0.3/1M tokens
Output$1.5/1M tokens

How Command A+ ranks now

BenchmarkScoreRankSource
Arena Elo1354 Elo115 of 141LMArena, Nov 2026 ↗
MMMU75.1%10 of 13llm-stats, Jul 2026 ↗

Rank is against every model in our database with a sourced score on that benchmark, each counted once with its best score.

All Command A+ specs and scores →

Cohere Releases Command A+: 218B-Parameter MoE Model With 4-Bit Quantization Runs on Single B200 GPU

Cohere has released Command A+, an open-source sparse mixture-of-experts (MoE) model with 218 billion total parameters and 25 billion active parameters. The model's W4A4 quantization enables deployment on a single Nvidia B200 GPU, significantly reducing hardware requirements compared to full-precision variants.

Technical Specifications

Command A+ uses a decoder-only architecture with 128 experts, activating 8 per token plus one shared expert. The model supports:

  • Context length: 128K input, 64K output
  • Quantization options: BF16 (requires 4x B200 or 8x H100), FP8 (2x B200 or 4x H100), W4A4 (1x B200 or 2x H100)
  • Languages: 48 languages including English, Chinese, Japanese, Korean, and major European and Asian languages
  • Modalities: Text and image inputs
  • License: Apache 2.0

Cohere states all three quantization levels show "negligible differences in benchmark quality," though specific benchmark scores were not disclosed in the model card.

Quantization Methodology

The W4A4 quantization applies NVFP4 4-bit precision to MoE experts only, while keeping attention layers (Q/K/V/O projections and KV cache) at full precision. According to Cohere, this selective approach addresses the "outsized quantization tax" that reasoning models typically face, where per-token errors compound during long decoding traces.

The company used Quantization-Aware Distillation (QAD) during post-training, training the quantized model to match the full-precision version's output distribution using fake quantization operators in forward passes and straight-through estimators on backward passes.

Architecture Details

The model interleaves sliding-window attention layers with rotational positional embeddings and global attention layers without positional embeddings in a 3:1 ratio, building on the architecture introduced in Command A. The sparse MoE layer uses a token-choice router with additive-bias-based load balancing and replaces the standard softmax activation with normalized sigmoid over top-k expert logits.

Command A+ includes native chain-of-thought reasoning, generating intermediate thinking steps between <START_THINKING> and <END_THINKING> tags. The model also supports conversational tool use through JSON schema-based function calling integrated into the chat template.

Deployment Requirements

The W4A4 variant requires vLLM version 0.21.0 or higher and Cohere's melody library (version 0.9.0+) for accurate response parsing. Cohere recommends sampling parameters of temperature=0.9, top_p=0.95, and repetition_penalty=1.04.

Pricing information for API access has not been disclosed.

What This Means

Command A+'s ability to run on a single B200 GPU represents a meaningful reduction in deployment costs for models in the 200B+ parameter range. The selective quantization approach—preserving full precision in attention while compressing experts—suggests a practical path for maintaining reasoning quality while reducing memory footprint. However, without published benchmark scores, it's unclear how Command A+ compares to frontier models like GPT-4, Claude 3.5 Sonnet, or DeepSeek V3 on standard reasoning tasks. The model's support for 48 languages and tool use capabilities positions it for enterprise agentic applications, though real-world performance validation remains to be seen.

Related Articles

model release

Tencent releases Hy4 flash: 220B MoE, 12B active parameters, 1M-token context

Tencent has released Hy4 flash, a multimodal Mixture-of-Experts model with 220B total parameters, 12B active per token, and a 1,048,576-token context window. Tencent calls it its most cost-effective Hy model to date and claims a per-task cost of 1/20 that of Hy4 on its internal WorkIndex benchmark.

model release

Tencent releases Hy4 Flash Base: 220B-parameter MoE, 12B active, 1M-token context

Tencent has published Hy4 Flash Base on Hugging Face: a 220B-parameter mixture-of-experts model with 12B active parameters per token, text and image input, and a 1M-token context. Tencent calls it its most cost-effective Hy model to date.

model release

SenseNova-Loop-9B released: 9B parameters, 56 layer executions, 512K context

SenseNova has released SenseNova-Loop-9B, a roughly 9B-parameter multimodal model that runs 56 decoder-layer executions per forward pass from 32 distinct layers. It supports up to 512K tokens of context and, per the model card, needs about 15 GiB of GPU memory at 256K context with 4-bit weights and an 8-bit KV cache.

model release

RadixArk releases DaisyRose-27B, lifting Terminal-Bench 4.1 pass@1 from 5.6% to 12.0%

RadixArk released DaisyRose-27B, its first model, built on Qwen3.8-27B and trained with reinforcement learning on 1,200 curated agent tasks. According to RadixArk, it raises Terminal Bench 4.1 pass@1 from 5.6% to 12.0% and scores higher than the base model on all seven reported benchmarks. Weights ship under Apache 2.0 in BF16, FP8 and NVFP4.

Comments

Loading...