Trusting AI systems with real world responsibilities requires measuring performance.
First we build a system to make decisions, which is mostly collecting input data for the large language model (LLM). We work with you on optimizing the tradeoff between false negatives, the cases it should have caught and didn’t, and false positives, the cases it flagged that were fine, and what happens in each scenario.
Readiness map: ten business days, fee credited against the build.
LLMs answer confidently even when they're wrong, most often because inputs were incomplete or incorrect.
The old adage garbage in, garbage out applies. LLMs also provide random confidence levels on how sure they are. The percentages they provide are pretty random.
So it’s important to build failsafes to catch these situations as well as preventative measures to ensure the system has as much context as possible, similar to context you or your employees have when making decisions and taking actions, especially so for high stakes hard-to-reverse actions. Examples include: merging customer relationship management (CRM) records for potential duplicates, and sending emails to customers or prospects.
- Pulling out facts and comments from (imperfect) meeting transcripts
- Matching customers to records like invoices
- Updating a deal status ephemerally (like an ambient CRM, one that updates itself from your email and meetings)
Domain experts assess the decisions and correct them as needed.
First we build a system to make decisions, then we do a dry run where a frontier LLM makes the decisions for a representative sample of the total class of decisions. Domain experts (likely a combination of our and your team) assess the decisions. This becomes a “golden set” for our system to check against: the answers you know are right.
The review is vital for a few reasons: (1) it helps catch edge cases (2) it builds up and expands the golden set for future iterations. If done right, the review workload drops with time as we calibrate your system. If and when a decision is shown to a human, we give the person all relevant information, and if data is missing that’s a sign that the automated system is missing data or has incorrect data. Back to data collection, and we run the evals (the automated checks) against the updated golden set.
The humans that will review may end up being a combination of our team, your team and 3rd party contractors such as business process outsourcing (BPO) shops.
We decide on a specific numerical goal and iterate until we achieve it.
After we have initial performance data (false positive rate and false negative rate), we sit down and go through the numbers to understand them, discuss methods to improve them (collect more data, change the system logic, expand the golden set, etc). Then we monitor it over time.
When designing the system, we try to figure out if it’s a high stakes hard-to-reverse decision, high stakes easy-to-reverse decision, low stakes hard-to-reverse decision, or low stakes easy-to-reverse decision.
If a decision is high stakes hard-to-reverse, we can build in a grace period to prevent the decision from calcifying (e.g. don’t send emails right away), add post-decision human review, and other techniques to minimize damage. If easy to reverse, we can and should reverse it. If low stakes hard-to-reverse, we work with you on what should happen.
Either project based, goal based or on a per hourly basis.
The scope for an evals system such as this is critical for the readiness map because it’s a critical portion of the build and retainer. As part of the readiness review we identify scope and work with you on what works best for you.
Find out what your data is actually like before you budget for it.
I think it behooves every business today to examine where their data is and try to consolidate it in order for AI to do real work. I would ask what their current posture is with AI, with LLMs, and how they're using them, and what their alternative is. Ten business days, and the fee comes off the build if you book one within thirty days.
- Days one to fourSystems in, matches checked
Each system of record walked through, then a check of where your systems disagree about your top 500 clients, projects or matters
- Days five to tenVerdict, uses, price
Go or no-go per source on export rights, then a fixed price naming sources, records and tools
- Read access only, or a guided screen share
- Nothing changed in any source system
- Only counts appear in the document
- No training or benchmarking on your data
- Mutual nondisclosure agreement (NDA) before kickoff
- You keep the document, even if it says not to build