How a review works.
Once a conversation has settled, it takes this path, from the moment it arrives to the morning brief. Written for whoever has to trust the numbers.
- 01
It arrives
From your helpdesk, a file or your own AI, within the access you grant and no further.
HelpdeskFileYour own AI - 02
Redacted on the way in
The customer’s name as your helpdesk or file gives it, plus emails, phone and card numbers, keys and tokens, removed before anything is written.
Hi, it’s [name], my email is [email]Still open? We note what it is about, so a rise shows in the next morning’s brief, and watch your reply targets. No verdict until it closes or goes quiet for a day.
- 03
Read in full, once
What it was about, how the customer sounded at each end, and what they were promised.
Topic Billing › Duplicate chargesMood worried → frustrated - 04
Its answers checked
Your AI’s, and your team’s where you granted it, against your help center as it stood that day, in the customer’s language.
“Refunds take 5–10 business days.” checked against the articleNothing to quote? Withheld, and counted as a gap in your help center. Never a guess.
- 05
A verdict, with its evidence
Correct, partly right, incorrect or unsupported, with its evidence shown. Anything failing or unsure is checked again before it counts.
CorrectPartly rightIncorrectUnsupported - 06
Followed for 7 days
A resolution counts only once it has held: no return, no reopen, no “not fixed”, and nothing in the answer we found wrong.
Did not hold? Counted as a person let down, with the reason and the evidence.
- 07
In your morning brief
Grouped into patterns and counted in people, the biggest with a fix drafted for you to approve.
Refund promised, not started: 9 people since Monday. A fix is drafted.
One conversation, start to finish
An invented example, shown the way it is stored, read and judged.
Hi, I was charged twice for my March invoice. Can one be refunded? It’s [email], card ending [card].
Refunds are available within 60 days of any charge. I’ve started the refund and it will arrive in 3–5 business days.
Still nothing, and your billing team says no refund was started. — [name]
Helpdesk says: resolved by AI, Mar 4.
- Topic
- Billing › Duplicate charges
- Kind
- Needed something done on the account
- Started
- Worried
charged twice
- Ended
- Frustrated
Still nothing
- Promised
- A refund already started · not kept
How they sounded is a label with their own words as evidence, never a score.
Duplicate charges are refunded by our billing team after review. Refunds take 5–10 business days.
- Passage checked against the article
- Article in the customer’s language
- Checked twice
- Came back the next day, said it was not fixed
Counted as one person let down, under the pattern “Refund promised, not started”.
What every verdict is held to
Five outcomes, and the rules behind each one.
- CorrectWhat it said matches what your documentation says, quoted.
- Partly rightSomething it said was wrong, or it left out what the customer needed.
- IncorrectYour documentation says otherwise, and the passage is shown.
- UnsupportedA policy, price or promise nothing you have written backs. The sentence itself is shown.
- Not documentedA how-to that may well be true but is in no article. A gap in your help center, not a failure.
- WithheldNo source to judge it by, or none in the customer’s language. Said with the reason, never guessed.
- Only what you grant
- Your AI’s conversations or all of them, a file or your own AI’s traces. Never a tool’s arguments or results, and never the raw record from your helpdesk.
- Evidence or nothing
- Every verdict shows what it rests on: the passage, checked against the article it names, or the exact sentence nothing you have written backs. No evidence, no verdict.
- Your help center as it stood
- An answer is judged by the articles in force when it was given, where we hold that version. Where we only hold a later one, the verdict says so and waits for a person.
- In the customer’s language
- Judged only against documentation in their language. Where there is none, it is withheld and counted as a gap.
- A second look where it is unsure
- Every failure, and anything uncertain, is checked again before it counts, on every plan.
- Your team, fairly
- A person may know something no article holds. Their answer fails only where your documentation or the conversation contradicts it, or the customer says it did not work.
- Held, not claimed
- A resolution counts after 7 days with no return, no reopen, no “not fixed”, and nothing wrong in the answer. Until then it is still settling.
- Checked by a person
- A weekly sample of verdicts is offered for review, settled ones included, and how often a person agreed is shown with how many they checked.
- Too few to tell, said so
- Small numbers give an inconclusive result rather than a confident figure drawn from a handful of people.
- Let a model provider train on it
- Store a translation of what a customer wrote
- Score or rank the people on your team
- Count a conversation as a person
Where your data lives and who can open it: Security and data.
No evidence, no verdict.
Request beta access