THE CRUNCH
OpenAI has made its Decisions API widely available to all customers for the first time, formally entering the market for fast, cheap decision models. These models, sometimes called System One models, act as super classifiers: given a set of options, they quickly pick one and attach a confidence rating, letting developers replace slower, pricier large language models in many workflows.
The product lead for OpenAI's APIs, Nikunj Handa, told TechCrunch the team moved quickly after seeing TypeSafe's Jev model, which turned the startup into a $7.6 billion company. Rather than training a purpose-built decision model, OpenAI reconfigured its ChatGPT Luna model to speed up inference and cut costs, shrinking response times from one to three seconds down to 100, 120 or 180 milliseconds. The API was announced at OpenAI's developer day on September 29 and became available to all customers on Thursday, with new voice and visual modalities added and latency reduced by targeting inference resources geographically.
Early customers suggest the approach works. Speak, which builds AI language tutors, says the API cut the time between actions from 1.4 seconds to 0.13 seconds, and OpenAI used it to improve its own Dots voice assistant. Handa argues OpenAI's model has an edge in multi-modality, saying Luna is "extremely powerful" for image classification and shows better calibration, though customers want more flexibility around image outputs.
Public benchmarks tell a more mixed story. Indices such as the AIM System One Index and the Artificial Analysis Decision Index tend to favour Liquid's new D1 Ultra and Base Lab's Elephant-1, with OpenAI's model placing eighth and fourteenth respectively, while TypeSafe's latest Jev ranks seventh and third. TypeSafe's CEO, Diogo Almeida, distrusts benchmarks and urges developers to trial models in their own workloads, a fair caveat given that optimisation across intelligence, cost, calibration and speed appears to differ by task. On the lighter side, JevMan, a benchmark where models play Pac-Man, is topped by the latest Jev model.