DECISION MODELS
Aligning Jev and other decision models to your data
A decision model like Jev answers a bounded question with a verdict and a confidence, in a fraction of a second, for a fraction of a cent. It arrives frozen: you can’t fine-tune it, and it doesn’t know your reviewers’ rules. Everything that makes it fit your business lives in the loop around it. Each step of that loop has a published measurement behind it, and we run the loop on client judgment tasks.
- Question
- Yes
- No
A question goes in. A verdict comes out, with a confidence we can check.
01 / Questions
The questions people arrive with
Can you fine-tune Jev?
No. TypeSafe serves the same Jev weights to every account, so there is no fine-tuning endpoint. You get the same effect by changing the questions you ask it and fitting a small decision head over its answers, aligned from reviewer feedback. On 140 labels, one new plain-English question raised accuracy from 77 to 87 percent with the model unchanged.
How do I align Jev to my own data?
Start classifying immediately, review the decisions as it makes them, and label the ones a reviewer disagrees with. Those labels refit the decision head and calibrate its confidence, and when disagreements keep coming, they tell you which question to add. That loop is far cheaper than building a labelled dataset before you start.
Can you trust Jev’s confidence?
Higher really does mean more likely right, but out of the box it ran about 15 points overconfident on 8,801 labelled examples. Calibration against a few hundred of your own labels closes most of that gap, and only then can you set a threshold to auto-accept above and route the rest to a person.
Does a decision model change its verdict on a name or a pronoun?
It can. On two thousand real professional bios per job with one pronoun swapped, Jev’s verdict moved on 1 to 4 bios in 100 and the open-weights Laya’s on 8 to 18, almost always toward the stereotype. Auditing by slice before and after deployment is how you catch it.
Can I own the classifier instead of paying per request forever?
Yes. Once the hosted model is aligned, you can distill it into a small model you serve yourself. A 66-million-parameter student trained from an aligned Jev system scored 91.2 percent against the human label, above its teacher’s 89.0, at 5 to 15 milliseconds an item, with a per-slice ship gate deciding when it was ready. You own and keep everything we build for you. Our solutions are open source and stay open. Integration code specific to your project is yours and private, and anything it depends on in our public projects stays public.
Can I run this loop myself?
Yes. The loop is open source as Cyclotron, under the MIT license. A cyclotron is one AI decision that learns from your reviewers’ corrections: it starts on day one with the decision model’s general training, refits a small model on each correction, and lets an LLM propose better questions that code tests before they go live. Run it yourself, have Anthus run it, or have Anthus build it into your product.
Should I use Jev or an open-weights decision model like Laya?
The same reviewer loop aligns either one. On the same 140 labels and the same questions, Jev led Laya by 4.7 points alone and 6.8 with the feedback layer, and fully fine-tuning Laya on those labels beat both. Which engine you pick depends on whether you need to own the weights.
02 / The loop
The loop, in order
- Start classifying now
Put the decision model in front of real cases with the questions your reviewers already ask.
- Review and label the disagreements
Reviewers agree or disagree with verdicts as they happen, and those calls are all the labelling the loop needs.
- Calibrate the confidence
Fit the stated confidence to observed accuracy so a threshold means what it says.
- Change the questions when refits stall
Let the system read the disagreements and propose one question; a person approves it.
- Gate every change by slice
Promote a version only when it clears held-out labels on every slice reviewers care about.
- Distill when you’re ready to own it
Train a small model from the aligned system and ship it behind the same gate.
Every step is a published experiment on the research page, the public leaderboards are on the benchmarks page, and Primus is where we run the loop in production.
PRESS START
We do this for clients, at scale.
Anthus has run this kind of loop in production for years: reviewers correct the model and say why, the explanation becomes policy, and the system gets more trustworthy month over month across hundreds of scorecards and millions of interactions. Bring us the judgment task and we’ll run it with your reviewers in the loop, and hand you a report you can inspect after the first month. The engine is open source: Cyclotron.