inference speed

OpenAI has launched a preview of 'Ultrafast' mode for GPT-5.6 Sol, delivering up to 750 output tokens per second through Cerebras inference hardware. The feature is initially limited to select API customers as part of a tiered speed pricing structure.

OpenAI has introduced 'Ultrafast,' a preview mode that it claims accelerates GPT 5.6 Sol to 14 times standard speed, hitting up to 750 output tokens per second. The feature runs on OpenAI's partnership with chipmaker Cerebras and is currently limited to a small group of customers.

OpenAI is testing an 'Ultrafast' service tier that runs GPT-5.6 Sol up to 14 times faster than standard processing, generating up to 750 output tokens per second using Cerebras infrastructure. Access is currently limited to a waitlist of select customers.

Nvidia's Nemotron 3.5 Lightning, a 31.6B-parameter open-weight model with only 3.6B active parameters, matches OpenAI's gpt-oss-120b on the Artificial Analysis Intelligence Index while delivering the fastest throughput in its class at nearly 670 tokens per second. The model posts especially large gains on agentic benchmarks, beating both gpt-oss-120b and the larger Nemotron 3 Super.

Google DeepMind released DiffusionGemma, a 26B parameter mixture-of-experts model that generates text using discrete diffusion instead of autoregression. The model processes blocks of 256 tokens in parallel, achieving generation speeds exceeding 1100 tokens per second on H100 GPUs in low-batch settings.