20 cases · three runs per model
Support routing and escalation cases
Distinguish an outage affecting all customers from one team’s problem, a billing question or a sales request.
Historical records from October 8, 2026. Synthetic cases with ClefBench references, not independently reviewed. No new model calls.
support-01
Our production API is down for every customer. No requests succeed.
support-02
Please send a copy of our last paid invoice. Everything works.
support-03
We need a quote for twenty seats next quarter.
support-04
One developer cannot use a new export feature. CSV download still works.
support-05
The payment page rejects cards for all customers. Renewals are blocked.
support-06
Our entire finance team cannot open invoices today. Other services work.
support-07
Can we arrange a demonstration next week?
support-08
All users in our company cannot log in. Other companies are unaffected.
support-09
There is a small duplicate charge on my account; work is unaffected.
support-10
Production traffic is failing globally after the release.
support-11
Do you offer volume discounts for next year?
support-12
One receipt has the wrong address; I can still download it.
support-13
A typo in the help page needs correction. No function is affected.
support-14
Every tenant has lost access to stored records; there is no workaround.
support-15
Procurement needs an enterprise quote by tomorrow, but operations are normal.
support-16
A single user sees a broken avatar; core functions still work.
support-17
Our whole department cannot export any report, but can view them.
support-18
All organizations were charged ten times the agreed amount today.
support-19
Please explain the annual price before we buy.
support-20
Everyone in our company sees incorrect invoice totals and cannot pay. Other customers are fine.
Browse benchmark cases by task
Support routing and escalation cases →
Distinguish an outage affecting all customers from one team’s problem, a billing question or a sales request.
Invoice classification and review cases →
Inspect expense categories, missing invoice fields and the difference between unverified suppliers and confirmed fraud.
Risk triage and severity cases →
Compare active incidents with blocked attacks, contained hazards and routine checks that require no action.