ChatJudge sample report on synthetic documents · run 2026-11-15 22:42 SGT

Sample report: which model extracts these invoice fields, and what does it cost?

Which model extracts the fields you need, and what does it cost per document? Tested on the same five documents and the same field list for all three models.

Decision

Claude Opus 5.5, GPT-6 Astra and Gemini 4 Carbon all captured every field you marked essential on these five documents, so they tie on your requirements.

Of those, Gemini 4 Carbon costs least, at $0.0081 per document against $0.0184 for Claude Opus 5.5 and $0.0239 for GPT-6 Astra.

This applies to these five documents only. It is not a general ranking of the models.

Essential fields

Essential fieldClaude Opus 5.5GPT-6 AstraGemini 4 Carbon
Invoice number5/55/55/5
Invoice date5/55/55/5
Supplier name5/55/55/5
Purchase-order number5/55/55/5
Total amount due5/55/55/5
All essential values25/25 · met25/25 · met25/25 · met

Overall accuracy and cost

ModelAll values rightMissedWrongUnexpectedAmbiguousCost per documentCost, 5 documentsTime, 5 calls
Claude Opus 5.5
Anthropic · claude-opus-5-5
45/45 (100%)0000$0.0184$0.091832.1 s
GPT-6 Astra
OpenAI · gpt-6-astra
45/45 (100%)0000$0.0239$0.119323.7 s
Gemini 4 Carbon
Google · gemini-4-carbon
45/45 (100%)0000$0.0081$0.040445.7 s

Cost = tokens reported by the gateway × list price per token (claude-opus-5-5: $4.40 in / $22.00 out per million tokens; gpt-6-astra: $11.00 in / $55.00 out per million tokens; gemini-4-carbon: $2.20 in / $11.00 out per million tokens). Marked unknown when the gateway reported no usage.

Representative errors

No errors: every model got every value right.

Per-field results

FieldTypeClaude Opus 5.5GPT-6 AstraGemini 4 Carbon
Invoice number (essential)Identifier5/55/55/5
Invoice date (essential)Date5/55/55/5
Due dateDate5/55/55/5
Supplier name (essential)Name5/55/55/5
Customer nameName5/55/55/5
Purchase-order number (essential)Identifier5/55/55/5
SubtotalAmount with currency5/55/55/5
Tax amountAmount with currency5/55/55/5
Total amount due (essential)Amount with currency5/55/55/5

Method

ChatJudge · chatjudge.ai · Advark Tech Pte Ltd, Singapore.