AI ENGINEERING · SINCE 2007
AI that keeps working after the demo.
We build custom models, agent systems and knowledge bases, run them in production, and keep them aligned with the people who know the work.
- Confident: auto
- Uncertain: ask a person
- Reviewer says no
- Correction
Confident items go straight through. Uncertain ones go to a person, and every correction travels back to the model.
- $250MProcessed in production
- 180BTokens of production LLM work
- ~100%Uptime
- 2007Building production systems since
01 / What we do
Four kinds of system. One way of running them.
Clients usually arrive asking about one of these. The work rarely stays in one box.
- Docs
- Calls
- Rules
- Found
Scattered material sorts itself into a taxonomy, so a query lands on the right block.
Knowledge Bases
Your agents are only as good as what they can look up. We build knowledge bases with ontologies and taxonomies that learn from your data, instead of going stale the week after someone hand-built them.
- Reviewer says no
- Rule
- Passed by rule
A reviewer says no once, and we write down why. After that, the rule handles the same case without them.
Self-Aligning Automation
Business process automation with a human in the loop. Reviewers correct it and say why; the system turns the explanation into a stated policy and applies it from then on.
- Agent
- Tool
- Limit
- Goal
The harness bounds the agent: it can reach its tools and its goal, and nothing outside the limit.
Agent Systems
A model is half a system. The other half is the harness: the tools it can reach, the procedures it follows, the limits it runs inside, and the ability to work for hours without losing the plot.
- Above the bar: auto
- Below: escalate
A calibrated model knows when it is unsure: above the bar it decides, below it asks a person.
Machine Learning
Custom classifiers, fine-tuned models, and decision models like Jev, with calibrated confidence that tells you which decisions to trust and which to escalate. We find the cheapest model that clears your bar and run it on AWS.
02 / Proof
Systems that run for years, not weeks.
- Pass
- Miss
Each call is one document and gets one scored row. We have scored calls this way across hundreds of scorecards and millions of calls.
Call Criteria
Call-center QA, scored by an outside service’s fine-tuned classifiers.
- Problem
- Too expensive and too slow to scale to every new scorecard.
- What we did
- We built their classifier lab on Primus in their own AWS account and ran it for years.
- Outcome
- Hundreds of scorecards, millions of calls scored.
- Event
- Function
- Delivered
Ticketing, VIP tables and guest lists on serverless, event-driven infrastructure, wired into loyalty, ServiceNow and a data warehouse.
Venue Driver
Ticketing, VIP table reservations and guest lists for Las Vegas nightlife.
- What it was
- A bespoke, serverless, event-driven system (AWS Lambda, Amplify, SQS), operationally mature.
- What it integrated
- Integrated with Tao Group’s in-house loyalty system, Madison Square Garden Entertainment’s ServiceNow ITSM, and a bespoke data warehouse and business intelligence system.
- Outcome
- In continuous operation since 2007.
03 / Solutions
Solutions we built from the work.
Our in-house solutions, from a stack that grew out of real delivery work.
- Reviewer says no
- Rule
- Passed by rule
A reviewer says no once, and we write down why. After that, the rule handles the same case without them.
Cyclotron
One AI decision that learns from your team’s corrections.
- Approved
- Declined
- Needs approval
People and bots share one room. A bot asks before it acts: a person approves or declines each request.
Chatticus
Shared rooms where people and bots work together, with approvals.
- Approved
- Held
- Agent
- Editor
Agents watch the beat, assign, report and draft. A person, the executive editor, approves each item before it publishes.
Papyrus
An automated newsroom: agents watch your beat, research and draft, and your editors decide what publishes.
- Question
- Yes
- No
A question goes in. A verdict comes out, with a confidence we can check.
Primus
The classifier lab: evaluate, deploy and improve AI with reviewer feedback.
Supporting technology Kanbus Tactus Biblicus Auritus Limatus Pudicus
04 / Articles
Latest from the lab.
It's Not a System 1 Model If It Takes Ten Seconds
Fastino's GLiDE beats every open model on our reasoning benchmark, but not Jev: less accurate, five times slower, ten times the cost, and a confidence that barely tells right from wrong.
Can Averaging Reduce Noise in an AI Decision Model?
We asked Jev the same question ten times in one request and expected ten identical answers. They split, four votes to six. Kahneman, Sibony and Sunstein call that kind of variation noise. Averaging the copies didn't make Jev more accurate, but it cut changed answers on a rerun by about a third.
- Question
- Yes
- No
A question goes in. A verdict comes out, with a confidence we can check.
The OpenAI Decisions API Needs a Confidence You Can Trust
We've routed decisions on LLM confidence since 2023, and calibration is what we most hope OpenAI's new Decisions API gets right. The model underneath, GPT-6 Luna, isn't there yet: on 3,600 reasoning problems its 99% meant 68%, and by five inference steps it was at a coin flip.
- Reviewer says no
- Rule
- Passed by rule
A reviewer says no once, and we write down why. After that, the rule handles the same case without them.
Primus Is a Classifier Lab: Operationalizing Classifiers at Scale
A hosted decision model sat at 77 percent accuracy through 87 rounds of reviewer feedback, then one approved question took it to 87. Here is the loop that did it, every figure published, and the offer to run it on your judgment task.
Decision Models Are Not Calculators
Jev chose one in every ordering of a fair-die question; Kev leaned toward one, and Laya overwhelmingly chose six. Benford's Law remains a hypothesis.
Can You Fix It? Gating, Averaging and Fine-Tuning Against a Gendered Verdict
A decision model's verdict moves on "she". We put a gate on the learning loop that rejects any question whose answer moves with the pronoun. It kept out the obvious ones and still made a hiring shortlist worse, because a question can ignore the pronoun and read gender from the words. Here's what worked instead.
PRESS START
Bring us the problem.
Even if you can’t yet say what you need. Working that out is part of the first month, and it’s often the most useful part.
