OpenAI opens Decisions API to all customers, cutting latency to 100-180 ms on a reconfigured ChatGPT Luna

OpenAI made its Decisions API available to all customers on Thursday, according to TechCrunch. The service runs ChatGPT Luna reconfigured for faster inference and lower cost, and OpenAI says it cuts response times from one to three seconds down to 100, 120 or 180 milliseconds.

The API was announced at OpenAI's developer day on September 29. It targets the category of "decision" or "System One" models that TypeSafe's Jev established: deep learning models that act as super classifiers, choosing among given options and returning a confidence rating. TypeSafe, founded by a former researcher at OpenAI, is now valued at $7.6 billion.

What OpenAI shipped

Nikunj Handa, product lead for OpenAI's APIs, told TechCrunch the team did not build a specifically calibrated decision model. "Right now, it's just plain old Luna that gets the job done," he said, adding that OpenAI may add specific training in the future.

The public version adds voice and visual modalities. OpenAI also targeted its inference resources geographically to lower latency in regions around the world. Pricing, context window and parameter count were not disclosed.

Customer results

Box, Datadog, Ramp and Speak took part in initial trials, replacing OpenAI LLMs in their workflows for higher throughput. Andrew Hsu, CTO of Speak, said the API cut the time between actions from 1.4 seconds to .13 seconds. OpenAI also used the product to improve the responsiveness of its Dots voice assistant.

Benchmark standing

Public rankings place OpenAI's decision model behind several competitors:

• AIM System One Index: the latest Jev model ranks seventh; OpenAI's model ranks eighth.
• Artificial Analysis Decision Index: the latest Jev model ranks third; OpenAI's model ranks fourteenth.
• Both indices tend to favor D1 Ultra, released by Liquid last week, and Base Lab's Elephant-1.
• TypeSafe's models are currently the most widely queried on OpenRouter.

The latest Jev model, released last week, adds support for images and smart-if statements. It also tops JevMan, a benchmark built by Opper AI in which decision models play PacMan. Felix Wunderlich, Opper AI's chief growth officer, said the game tests a nonstop stream of high-stakes decisions, with a few hundred milliseconds at each junction.

TypeSafe CEO Diogo Almeida distrusts benchmarks and urges users to trial models on their own work, though TypeSafe published its own rankings alongside the new Jev release. TechCrunch notes that the right balance of intelligence, cost, calibration and speed appears to differ by workload.

Where OpenAI sees an edge

Handa said multimodality is where OpenAI's model may have an advantage. For image classification, he said, "Luna is extremely powerful and has been showing better calibrations." He added that customers want the API to be more flexible about using images as outputs. He also floated a latency target of about 60 milliseconds, which would make a call on every frame of a video feasible.

What this means

OpenAI's entry is a repackaging of an existing model, not a purpose-trained decision model, so its current gains come from inference configuration and regional serving. On the two public indices cited, it ranks below the latest Jev model, while OpenAI points to image calibration as its strength. With no pricing disclosed and rankings that vary by index, the practical test is the one Almeida and Handa both imply: run the candidate models on your own workload and measure latency, cost and calibration there.