Insight
What Should an AI Evaluation SLA Include
Capacity, process, reporting, and remedies—not a fake accuracy percentage disconnected from a rubric.
Published 2026-08-26 · Written by the Stratum editorial team. No individual author or outside clinical or legal reviewer is named for this article.
An AI evaluation SLA should describe capacity, quality process, and communication—not a fake accuracy percentage. “99 percent accurate” without a rubric and a gold set is not an SLA.
SLA checklist
- Seats and weekly hours the vendor commits to staff against the profile.
- Start window and replacement time for a vacated seat.
- Qualification and calibration required before production hours count.
- Review mix: single, dual, adjudication path.
- Gold-set monitoring cadence, without inventing a target you cannot measure yet.
- Reporting contents and frequency.
- Escalation owner on both sides.
- Security and access revocation at the end of the program.
- What happens if the rubric changes mid-flight.
- Commercial remedies that match the process, not a marketing penalty clause.
What to leave out
Do not copy an uptime SLA from software hosting. Reviewers are not an API. Do not require certifications this site does not claim. Do not treat inter-annotator agreement as a warranty without defining the item class.