HISTORICAL RECORD · 2026-10-08 · invoice-11
Flight reimbursement $540 with complete invoice details.
Flight reimbursement $540 with complete invoice details.
Synthetic case. References were set by ClefBench and have not received an independent second review. These are recorded results, not a new model call.
urgent
Does this invoice need review because any required field is missing, supplier or banking details are unverified, or fraud is confirmed?
Reference: false
| Model | Run 1 | Run 2 | Run 3 | Changed answer? |
|---|---|---|---|---|
| Clef | false | false | false | No |
| Clef-flash | false | false | false | No |
| Jev 1.13 | false | false | false | No |
Yes/no uses a 50% threshold. Score values are weighted scale indices; error flags and stability use the nearest level. Use the rule and full distribution to judge whether a difference matters for your task.
category
Classify the purchased item.
Reference: travel
- hardware: Physical computers, printers, monitors or servers
- software: Software licenses, applications or cloud storage subscriptions
- services: Professional or contractor work
- travel: Transport, accommodation or taxi expenses
| Model | Run 1 | Run 2 | Run 3 | Changed answer? |
|---|---|---|---|---|
| Clef | travel | travel | travel | No |
| Clef-flash | travel | travel | travel | No |
| Jev 1.13 | travel | travel | travel | No |
Yes/no uses a 50% threshold. Score values are weighted scale indices; error flags and stability use the nearest level. Use the rule and full distribution to judge whether a difference matters for your task.
severity
Rate invoice risk from the stated evidence.
Reference: 0
- 0: Complete invoice with no reported issue
- 1: A required invoice field is missing
- 2: Supplier identity or banking details are unverified
- 3: Fraud, forgery or fabrication is confirmed
| Model | Run 1 | Run 2 | Run 3 | Changed answer? |
|---|---|---|---|---|
| Clef | 0.1362 | 0.1362 | 0.1362 | No |
| Clef-flash | 0.234 | 0.234 | 0.234 | No |
| Jev 1.13 | 0.26 | 0.37 | 0.29 | No |
Yes/no uses a 50% threshold. Score values are weighted scale indices; error flags and stability use the nearest level. Use the rule and full distribution to judge whether a difference matters for your task.
Time and provider cost for this case
- Clef · run 1: 636 ms · $0.00009912
- Clef-flash · run 1: 428 ms · $0.00003717
- Jev 1.13 · run 1: 358 ms · $0.00001966
- Clef-flash · run 2: 374 ms · $0.00003717
- Jev 1.13 · run 2: 338 ms · $0.00001966
- Clef · run 2: 611 ms · $0.00009912
- Jev 1.13 · run 3: 387 ms · $0.00001966
- Clef · run 3: 757 ms · $0.00009912
- Clef-flash · run 3: 492 ms · $0.00003717
Full task, reference answers and raw recorded responses
{
"sample": {
"state": "Flight reimbursement $540 with complete invoice details.",
"questions": {
"urgent": {
"type": "noul",
"instructions": "Does this invoice need review because any required field is missing, supplier or banking details are unverified, or fraud is confirmed?"
},
"category": {
"type": "choice",
"instructions": "Classify the purchased item.",
"criteria": {
"hardware": "Physical computers, printers, monitors or servers",
"software": "Software licenses, applications or cloud storage subscriptions",
"services": "Professional or contractor work",
"travel": "Transport, accommodation or taxi expenses"
}
},
"severity": {
"type": "score",
"instructions": "Rate invoice risk from the stated evidence.",
"criteria": [
"Complete invoice with no reported issue",
"A required invoice field is missing",
"Supplier identity or banking details are unverified",
"Fraud, forgery or fabrication is confirmed"
]
}
}
},
"reference": {
"urgent": false,
"category": "travel",
"severity": 0
},
"runs": [
{
"caseId": "invoice-11",
"repeat": 1,
"outcome": {
"model": "clef",
"measuredAt": "2026-10-08T09:26:40.102Z",
"durationMs": 636,
"status": "success",
"result": {
"id": "gen-dec-1791451600-D5NB57ct76JL7yMGjdnx",
"model": "cloudflare/clef",
"provider": "Cloudflare",
"answers": {
"urgent": {
"type": "noul",
"noul": 0.0132
},
"category": {
"type": "choice",
"choice": "travel",
"probabilities": {
"hardware": 0.003,
"software": 0.0031,
"services": 0.0071,
"travel": 0.9868
},
"confidence": 0.9652
},
"severity": {
"type": "score",
"score": 0.1362,
"probabilities": {
"0": 0.9277,
"1": 0.0199,
"2": 0.041,
"3": 0.0114
},
"confidence": 0.817,
"legend": {
"0": "Complete invoice with no reported issue",
"1": "A required invoice field is missing",
"2": "Supplier identity or banking details are unverified",
"3": "Fraud, forgery or fabrication is confirmed"
}
}
},
"usage": {
"input_tokens": 413,
"output_tokens": 0,
"cost": 0.00009912
}
}
}
},
{
"caseId": "invoice-11",
"repeat": 1,
"outcome": {
"model": "clef-flash",
"measuredAt": "2026-10-08T09:26:40.747Z",
"durationMs": 428,
"status": "success",
"result": {
"id": "gen-dec-1791451600-yHLpVSKOe4bT6XNwk1qI",
"model": "cloudflare/clef-flash",
"provider": "Cloudflare",
"answers": {
"urgent": {
"type": "noul",
"noul": 0.0325
},
"category": {
"type": "choice",
"choice": "travel",
"probabilities": {
"hardware": 0.003,
"software": 0.0033,
"services": 0.0141,
"travel": 0.9796
},
"confidence": 0.9464
},
"severity": {
"type": "score",
"score": 0.234,
"probabilities": {
"0": 0.8618,
"1": 0.0569,
"2": 0.0668,
"3": 0.0145
},
"confidence": 0.6674,
"legend": {
"0": "Complete invoice with no reported issue",
"1": "A required invoice field is missing",
"2": "Supplier identity or banking details are unverified",
"3": "Fraud, forgery or fabrication is confirmed"
}
}
},
"usage": {
"input_tokens": 413,
"output_tokens": 0,
"cost": 0.00003717
}
}
}
},
{
"caseId": "invoice-11",
"repeat": 1,
"outcome": {
"model": "jev",
"measuredAt": "2026-10-08T09:26:41.183Z",
"durationMs": 358,
"status": "success",
"result": {
"id": "gen-dec-1791451601-JLPikEaHF49wwbDa952k",
"model": "typesafe/jev-1.13-20260917",
"provider": "TypeSafe",
"answers": {
"urgent": {
"type": "noul",
"noul": 0.29
},
"category": {
"type": "choice",
"choice": "travel",
"probabilities": {
"software": 0,
"hardware": 0,
"travel": 1,
"services": 0
},
"confidence": 1
},
"severity": {
"type": "score",
"score": 0.26,
"probabilities": {
"0": 0.85,
"1": 0.04,
"2": 0.11,
"3": 0
},
"confidence": 0.74,
"legend": {
"0": "Complete invoice with no reported issue",
"1": "A required invoice field is missing",
"2": "Supplier identity or banking details are unverified",
"3": "Fraud, forgery or fabrication is confirmed"
}
}
},
"usage": {
"input_tokens": 468,
"output_tokens": 75,
"cost": 0.000019656
}
}
}
},
{
"caseId": "invoice-11",
"repeat": 2,
"outcome": {
"model": "clef-flash",
"measuredAt": "2026-10-08T09:28:10.533Z",
"durationMs": 374,
"status": "success",
"result": {
"id": "gen-dec-1791451690-AMTDmUybbSbgUqM4zQsx",
"model": "cloudflare/clef-flash",
"provider": "Cloudflare",
"answers": {
"urgent": {
"type": "noul",
"noul": 0.0325
},
"category": {
"type": "choice",
"choice": "travel",
"probabilities": {
"hardware": 0.003,
"software": 0.0033,
"services": 0.0141,
"travel": 0.9796
},
"confidence": 0.9464
},
"severity": {
"type": "score",
"score": 0.234,
"probabilities": {
"0": 0.8618,
"1": 0.0569,
"2": 0.0668,
"3": 0.0145
},
"confidence": 0.6674,
"legend": {
"0": "Complete invoice with no reported issue",
"1": "A required invoice field is missing",
"2": "Supplier identity or banking details are unverified",
"3": "Fraud, forgery or fabrication is confirmed"
}
}
},
"usage": {
"input_tokens": 413,
"output_tokens": 0,
"cost": 0.00003717
}
}
}
},
{
"caseId": "invoice-11",
"repeat": 2,
"outcome": {
"model": "jev",
"measuredAt": "2026-10-08T09:28:10.920Z",
"durationMs": 338,
"status": "success",
"result": {
"id": "gen-dec-1791451690-d0J7J99HeYcEhBDaSuKj",
"model": "typesafe/jev-1.13-20260917",
"provider": "TypeSafe",
"answers": {
"urgent": {
"type": "noul",
"noul": 0.35
},
"category": {
"type": "choice",
"choice": "travel",
"probabilities": {
"travel": 1,
"services": 0,
"hardware": 0,
"software": 0
},
"confidence": 1
},
"severity": {
"type": "score",
"score": 0.37,
"probabilities": {
"0": 0.79,
"1": 0.05,
"2": 0.16,
"3": 0
},
"confidence": 0.63,
"legend": {
"0": "Complete invoice with no reported issue",
"1": "A required invoice field is missing",
"2": "Supplier identity or banking details are unverified",
"3": "Fraud, forgery or fabrication is confirmed"
}
}
},
"usage": {
"input_tokens": 468,
"output_tokens": 75,
"cost": 0.000019656
}
}
}
},
{
"caseId": "invoice-11",
"repeat": 2,
"outcome": {
"model": "clef",
"measuredAt": "2026-10-08T09:28:11.271Z",
"durationMs": 611,
"status": "success",
"result": {
"id": "gen-dec-1791451691-pbYWaAZsfn6uynmSsNZN",
"model": "cloudflare/clef",
"provider": "Cloudflare",
"answers": {
"urgent": {
"type": "noul",
"noul": 0.0132
},
"category": {
"type": "choice",
"choice": "travel",
"probabilities": {
"hardware": 0.003,
"software": 0.0031,
"services": 0.0071,
"travel": 0.9868
},
"confidence": 0.9652
},
"severity": {
"type": "score",
"score": 0.1362,
"probabilities": {
"0": 0.9277,
"1": 0.0199,
"2": 0.041,
"3": 0.0114
},
"confidence": 0.817,
"legend": {
"0": "Complete invoice with no reported issue",
"1": "A required invoice field is missing",
"2": "Supplier identity or banking details are unverified",
"3": "Fraud, forgery or fabrication is confirmed"
}
}
},
"usage": {
"input_tokens": 413,
"output_tokens": 0,
"cost": 0.00009912
}
}
}
},
{
"caseId": "invoice-11",
"repeat": 3,
"outcome": {
"model": "jev",
"measuredAt": "2026-10-08T09:29:40.903Z",
"durationMs": 387,
"status": "success",
"result": {
"id": "gen-dec-1791451780-aZfzOff0pNBlYhkIw5IJ",
"model": "typesafe/jev-1.13-20260917",
"provider": "TypeSafe",
"answers": {
"urgent": {
"type": "noul",
"noul": 0.31
},
"category": {
"type": "choice",
"choice": "travel",
"probabilities": {
"hardware": 0,
"software": 0,
"travel": 1,
"services": 0
},
"confidence": 1
},
"severity": {
"type": "score",
"score": 0.29,
"probabilities": {
"0": 0.84,
"1": 0.03,
"2": 0.13,
"3": 0
},
"confidence": 0.71,
"legend": {
"0": "Complete invoice with no reported issue",
"1": "A required invoice field is missing",
"2": "Supplier identity or banking details are unverified",
"3": "Fraud, forgery or fabrication is confirmed"
}
}
},
"usage": {
"input_tokens": 468,
"output_tokens": 75,
"cost": 0.000019656
}
}
}
},
{
"caseId": "invoice-11",
"repeat": 3,
"outcome": {
"model": "clef",
"measuredAt": "2026-10-08T09:29:41.306Z",
"durationMs": 757,
"status": "success",
"result": {
"id": "gen-dec-1791451781-sGKDlw27GsGtzqmKtC0V",
"model": "cloudflare/clef",
"provider": "Cloudflare",
"answers": {
"urgent": {
"type": "noul",
"noul": 0.0132
},
"category": {
"type": "choice",
"choice": "travel",
"probabilities": {
"hardware": 0.003,
"software": 0.0031,
"services": 0.0071,
"travel": 0.9868
},
"confidence": 0.9652
},
"severity": {
"type": "score",
"score": 0.1362,
"probabilities": {
"0": 0.9277,
"1": 0.0199,
"2": 0.041,
"3": 0.0114
},
"confidence": 0.817,
"legend": {
"0": "Complete invoice with no reported issue",
"1": "A required invoice field is missing",
"2": "Supplier identity or banking details are unverified",
"3": "Fraud, forgery or fabrication is confirmed"
}
}
},
"usage": {
"input_tokens": 413,
"output_tokens": 0,
"cost": 0.00009912
}
}
}
},
{
"caseId": "invoice-11",
"repeat": 3,
"outcome": {
"model": "clef-flash",
"measuredAt": "2026-10-08T09:29:42.079Z",
"durationMs": 492,
"status": "success",
"result": {
"id": "gen-dec-1791451782-ETWRjZm33QxELgyEw2vG",
"model": "cloudflare/clef-flash",
"provider": "Cloudflare",
"answers": {
"urgent": {
"type": "noul",
"noul": 0.0325
},
"category": {
"type": "choice",
"choice": "travel",
"probabilities": {
"hardware": 0.003,
"software": 0.0033,
"services": 0.0141,
"travel": 0.9796
},
"confidence": 0.9464
},
"severity": {
"type": "score",
"score": 0.234,
"probabilities": {
"0": 0.8618,
"1": 0.0569,
"2": 0.0668,
"3": 0.0145
},
"confidence": 0.6674,
"legend": {
"0": "Complete invoice with no reported issue",
"1": "A required invoice field is missing",
"2": "Supplier identity or banking details are unverified",
"3": "Fraud, forgery or fabrication is confirmed"
}
}
},
"usage": {
"input_tokens": 413,
"output_tokens": 0,
"cost": 0.00003717
}
}
}
}
]
}