On October 22, OpenAI officially opened up the Decisions API to all customers, introducing an API service for rapid decision-making scenarios, and participating directly in the competition for decision models driven by the Jev model of startup TypeSafe.
On September 29th, OpenAI announced this product. The core of it is reconfiguring the ChatGPT Luna model to reduce inference latency and usage costs, rather than specifically training an independent decision-making model.
According to Nikunj Handa, the product manager of the OpenAI API, the Decisions API can reduce processing time from 1–3 seconds to 100–180 milliseconds. The public version adds voice and visual modalities, and reduces latency by scheduling inference resources by region. In preliminary tests with customers such as Box, Datadog, Ramp, and Speak, this API increased processing throughput; among them, the language learning app Speak reduced the operation interval from 1.4 seconds to 0.13 seconds. OpenAI also uses it to improve the response speed of its Dots voice assistant.
Decision models can make choices quickly based on given options and provide confidence assessments, and are expected to replace some of the logical judgment processes of traditional large language models. Currently, products such as Jev, Liquid’s D1Ultra, and Base Labs’ Elephant-1 are all competing in this field.
In the public benchmark tests, the latest Jev model ranked seventh and third in two decision model indices, while the OpenAI model ranked eighth and fourteenth. As multimodal capabilities and inference speed continue to improve, decision models are expected to expand into application scenarios such as real-time interaction, image classification, and frame-by-frame video processing. However, different tasks still have varying requirements for speed, cost, and accuracy.