- Reviewer says no
- Rule
- Passed by rule
A reviewer says no once, and we write down why. After that, the rule handles the same case without them.
Primus Is a Classifier Lab: Operationalizing Classifiers at Scale
A hosted decision model sat at 77 percent accuracy through 87 rounds of reviewer feedback, then one approved question took it to 87. Here is the loop that did it, every figure published, and the offer to run it on your judgment task.
Can You Fix It? Gating, Averaging and Fine-Tuning Against a Gendered Verdict
A decision model's verdict moves on "she". We put a gate on the learning loop that rejects any question whose answer moves with the pronoun. It kept out the obvious ones and still made a hiring shortlist worse, because a question can ignore the pronoun and read gender from the words. Here's what worked instead.
Fine-Tuning Jev: You Can't. Here's What Gets You the Same Effect
You can't fine-tune Jev, or any hosted decision model: TypeSafe serves the same weights to everyone. So we kept it frozen and adapted the questions and a small fitted head around it. On 140 labels, one new plain-English question bought 10 points of accuracy, and it finds the hidden pattern about a quarter of the time.
Distilling an Aligned Jev System into a Classifier You Own
A Jev decision model aligned with 140 human labels taught a 66M-parameter DistilBERT that scored 0.912 against the human label, two points above its teacher, at 5.6 to 15 ms an item. The accuracy is the least general part; the per-slice ship gate is the part to copy.
Can You Trust Jev's Confidence?
A decision model's confidence should mean something. Jev's does: higher really does mean more likely right. But on 8,801 labeled examples they ran overconfident, and how you phrase the request changes the pattern. Calibration fixes most of it.
Never Use Fast
Premium Fast mode is a latency product, and agent work is almost never latency-bound. Do not confuse the expensive priority lane with the cheaper Flash-style models that made fast synonymous with efficient.
Maximize Value, Not Intelligence
In 2023, we argued for using the dumbest model the problem will bear. That rule still works, but choosing one model isn't enough anymore. Route each job to the cheapest model that clears the bar, then measure the cost of work you actually accept.
Maximize Profit, Not Intelligence
2024 won't just be about ever-smarter artificial intelligence. It will be about intelligently using artificial intelligence. Unless you're an AI researcher, your goal is to maximize profit. Not intelligence.
How AI Agents Do Things
How does it work when you ask an AI agent to draw a cute kitten and it draws a cute kitten? How did it learn to draw? Do AI researchers somehow upload new skills into the model like Neo in The Matrix?