product updateOpenAI

OpenAI launches GPT-Realtime-2 with GPT-5-class reasoning, adds real-time translation across 70 languages

TL;DR

OpenAI has added three voice intelligence features to its Realtime API: GPT-Realtime-2 with GPT-5-class reasoning for complex conversational requests, GPT-Realtime-Translate supporting 70 input languages and 13 output languages, and GPT-Realtime-Whisper for live speech-to-text transcription. Translation and transcription are billed by the minute, while GPT-Realtime-2 uses token-based pricing.

2 min read
1

OpenAI launches GPT-Realtime-2 with GPT-5-class reasoning, adds real-time translation across 70 languages

OpenAI has released three new voice intelligence features in its Realtime API: GPT-Realtime-2, a voice model with GPT-5-class reasoning; GPT-Realtime-Translate for real-time conversational translation; and GPT-Realtime-Whisper for live transcription.

GPT-Realtime-2: Voice with advanced reasoning

GPT-Realtime-2 succeeds GPT-Realtime-1.5 and includes what OpenAI describes as "GPT-5-class reasoning" designed to handle complex user requests during voice conversations. The model creates realistic vocal simulations and can converse with users while applying advanced reasoning capabilities. Pricing is token-based, though specific rates were not disclosed.

GPT-Realtime-Translate: 70 input languages, 13 output languages

The translation feature supports 70 input languages (languages it can understand) and 13 output languages (languages it can speak). According to OpenAI, the system provides real-time translation that "keeps pace" with conversational flow. The feature is billed by the minute, with pricing not yet disclosed.

GPT-Realtime-Whisper: Live transcription

GPT-Realtime-Whisper adds live speech-to-text capabilities, capturing transcriptions as conversations occur. Like the translation feature, it is billed by the minute.

Target use cases and safeguards

OpenAI positions these features for customer service, education, media, events, and creator platforms. The company stated it has implemented guardrails to prevent misuse for spam, fraud, or abuse. Conversations can be automatically halted if they violate OpenAI's harmful content guidelines, though the company did not specify how these triggers operate.

According to OpenAI, the new models move real-time audio "from simple call-and-response toward voice interfaces that can actually do work: listen, reason, translate, transcribe, and take action as a conversation unfolds."

What this means

The addition of GPT-5-class reasoning to voice models marks a capability upgrade beyond the previous generation, though OpenAI has not released GPT-5 itself or clarified what "GPT-5-class" specifically means in terms of benchmark performance. The 70-language translation support is substantial for multilingual applications, but the 13-output language limitation means many users will be able to understand the system but not receive responses in their native language. The per-minute billing for translation and transcription differs from the token-based model used for GPT-Realtime-2, which may affect cost predictability for developers building conversational applications.

Related Articles

changelog

OpenAI's GPT-5.6 Codex Bug Deletes User Files When Attempting to Override $HOME Environment Variable

OpenAI has identified a critical bug in GPT-5.6's Codex implementation that causes unexpected file deletions. According to Thibault Sottiaux, the issue occurs when the model attempts to override the $HOME environment variable to define a temporary directory but mistakenly deletes $HOME instead, particularly when full access mode is enabled without sandboxing protections.

product update

OpenAI restores chat sidebar in Mac app after user backlash over confusing redesign

OpenAI has updated its ChatGPT Mac app to restore direct access to chat conversations through a prominent sidebar toggle. The fix addresses user complaints following a July 10 redesign that replaced the native Mac client with an Electron-based app and buried the standard chat interface behind Work and Codex features.

changelog

Cline v4.0.9 Adds GPT-5.6 ChatGPT Models, Fixes Token Count Over-Reporting

Cline, the AI coding assistant VS Code extension, released v4.0.9 on July 16, 2024, adding support for GPT-5.6 ChatGPT subscription models. The update fixes a bug where token counts were over-reported from OpenAI-compatible providers due to improper handling of cumulative usage snapshots.

product update

Google prepares voice customization for Gemini with speed, energy, formality, and warmth controls

Google is preparing to let users customize Gemini's voice output across four parameters: speed, energy, formality, and warmth, according to code discovered in the Google app 17.41.12 beta. The controls will apply to both Gemini Live and standard chat interactions.

Comments

Loading...