It's Not a System 1 Model If It Takes Ten Seconds
Fastino's GLiDE beats every open model on our reasoning benchmark, but not Jev: less accurate, five times slower, ten times the cost, and a confidence that barely tells right from wrong.
- Question
- Yes
- No
A question goes in. A verdict comes out, with a confidence we can check.
The OpenAI Decisions API Needs a Confidence You Can Trust
We've routed decisions on LLM confidence since 2023, and calibration is what we most hope OpenAI's new Decisions API gets right. The model underneath, GPT-6 Luna, isn't there yet: on 3,600 reasoning problems its 99% meant 68%, and by five inference steps it was at a coin flip.
Decision Models Are Not Calculators
Jev chose one in every ordering of a fair-die question; Kev leaned toward one, and Laya overwhelmingly chose six. Benford's Law remains a hypothesis.
The One-Word Test: How Jev and Laya Read Gender, Race and Age
Two thousand real bios per job, first names blanked, one pronoun swapped, both decision models asked twice. Laya's verdict moved on 8 to 18 bios in 100, Jev's on 1 to 4, almost always toward the stereotype. Here is the method, the jobs, the shortlist, and what names and ages do.
We Told the AI She Was a Woman. It Demoted Her
A new kind of AI, the decision model, makes instant decisions about people: who gets an interview, whose claim gets flagged. It's cheap and fast, and companies are adopting it without checking what it does. We checked. It quietly ranks women lawyers below men, and every company using it makes the same mistake.
Jev vs Laya: Same Labels, Same Questions, One Variable
We ran two decision models, Jev and the open-weights Laya, through one harness where only the engine changes. Jev led by 4.7 points alone and 6.8 with our feedback layer, and then fine-tuning Laya on the same 140 labels beat both.
Decision Models: Making Decisions Instead of Generating Text
Jev is a decision model: it turns a scorecard’s bounded questions into the model’s native output, making high-volume QA faster, cheaper, and easier to operate.
How to Make a Text Classifier Using an LLM
Classification-with-Confidence follows an LLM classifier from Yes-or-No token probabilities through calibration, routing, and task-specific fine-tuning.
Maximize Profit, Not Intelligence
2024 won't just be about ever-smarter artificial intelligence. It will be about intelligently using artificial intelligence. Unless you're an AI researcher, your goal is to maximize profit. Not intelligence.
How AI Agents Do Things
How does it work when you ask an AI agent to draw a cute kitten and it draws a cute kitten? How did it learn to draw? Do AI researchers somehow upload new skills into the model like Neo in The Matrix?