Which AI works best on your team’s real tasks?
Build a private benchmark from the work you already do. Compare models on quality, cost, and speed. When a new model comes out, run the same test and see whether switching is worth it.
Create my benchmark See a sample benchmark
Free to set up. First comparison US$299.
Online payment is not switched on yet. Setting up is free, and you can send your first comparison as a request: nothing is charged until we confirm how to pay and the delivery date.
Meets your essential requirements: Claude Opus 5.5 (25/25 runs successful), GPT-6 Astra (25/25 runs successful) and Gemini 4 Carbon (25/25 runs successful).
Best quality: a tie between Gemini 4 Carbon, Claude Opus 5.5 and GPT-6 Astra (each met 100% of checks).
Lowest cost per successful task: Gemini 4 Carbon, $0.011 (failed attempts and retries included).
Fastest end to end: GPT-6 Astra, median 5.3 s per answer (9 in 10 within 9.4 s).
Case 1. Extend the initial term from 12 to 24 months. Keep the renewal term at 12 months and leave pricing unchanged.
Gemini 4 Carbon · run 1 · successful · 100% of checks · 14 s
This Services Agreement is made between Northwind Analytics Pte. Ltd. ("Supplier") and Harbourline Logistics Pte. Ltd. ("Customer").
1. Services. Supplier shall provide the route-analytics platform and support described in Schedule 1 (the "Services").
2. Term. This Agreement starts on the Effective Date and continues for an initial term of
3. Renewal. After the Initial Term, this Agreement renews automatically for successive renewal terms of twelve (12) months each, unless either party gives written notice of non-renewal at least thirty (30) days before the end of the then-current term.
4. Fees. Customer shall pay Supplier a subscription fee of four thousand eight hundred US dollars (US$4,800) per month, invoiced monthly in advance. The fees are fixed for the Initial Term. Invoices are payable within thirty (30) days of the invoice date.
5. Confidentiality. Each party shall keep the other party's Confidential Information confidential and use it only to perform this Agreement. This obligation survives for three (3) years after this Agreement ends.
6. Limitation of Liability. Except for breach of clause 5, each party's total liability under this Agreement is limited to the fees paid by Customer in the six (6) months before the event giving rise to the claim.
7. Termination for Convenience. Customer may terminate this Agreement for convenience after the Initial Term by giving Supplier thirty (30) days' written notice.
8. Governing Law. This Agreement is governed by the laws of England and Wales, and the courts of England and Wales have exclusive jurisdiction.
- The initial term is 24 monthsdealbreaker
Checked by code: answer matches the pattern /initial term of (twenty-four \(24\)|24|twenty-four) months/i - The renewal term stays at 12 monthsdealbreaker
Checked by code: this text is kept word for word: “3. Renewal. After the Initial Term, this Agreement renews automatically for successive ren…” - Pricing is unchangeddealbreaker
Checked by code: this text is kept word for word: “4. Fees. Customer shall pay Supplier a subscription fee of four thousand eight hundred US …”
Claude Opus 5.5 · run 1 · successful · 100% of checks · 11 s
This Services Agreement is made between Northwind Analytics Pte. Ltd. ("Supplier") and Harbourline Logistics Pte. Ltd. ("Customer").
1. Services. Supplier shall provide the route-analytics platform and support described in Schedule 1 (the "Services").
2. Term. This Agreement starts on the Effective Date and continues for an initial term of
3. Renewal. After the Initial Term, this Agreement renews automatically for successive renewal terms of twelve (12) months each, unless either party gives written notice of non-renewal at least thirty (30) days before the end of the then-current term.
4. Fees. Customer shall pay Supplier a subscription fee of four thousand eight hundred US dollars (US$4,800) per month, invoiced monthly in advance. The fees are fixed for the Initial Term. Invoices are payable within thirty (30) days of the invoice date.
5. Confidentiality. Each party shall keep the other party's Confidential Information confidential and use it only to perform this Agreement. This obligation survives for three (3) years after this Agreement ends.
6. Limitation of Liability. Except for breach of clause 5, each party's total liability under this Agreement is limited to the fees paid by Customer in the six (6) months before the event giving rise to the claim.
7. Termination for Convenience. Customer may terminate this Agreement for convenience after the Initial Term by giving Supplier thirty (30) days' written notice.
8. Governing Law. This Agreement is governed by the laws of England and Wales, and the courts of England and Wales have exclusive jurisdiction.
- The initial term is 24 monthsdealbreaker
Checked by code: answer matches the pattern /initial term of (twenty-four \(24\)|24|twenty-four) months/i - The renewal term stays at 12 monthsdealbreaker
Checked by code: this text is kept word for word: “3. Renewal. After the Initial Term, this Agreement renews automatically for successive ren…” - Pricing is unchangeddealbreaker
Checked by code: this text is kept word for word: “4. Fees. Customer shall pay Supplier a subscription fee of four thousand eight hundred US …”
Red is removed text, green is added. Each requirement shows how it was checked: by code where possible, otherwise by the grading model (GPT-6 Sol, gpt-6-sol, grader version tasks-judge-2026-11-16).
How it works
- Sign in with Google (or an emailed code). No organisation needed; share with your team later if you like.
- Choose the work to test: edit a contract, draft an agreement, extract information, or describe your own task. We suggest a name you can change.
- Add your cases. Paste a prompt and its input, pick a prompt your team already saved, or import a file. For each, say what a good response must get right and mark the dealbreakers. Five cases are enough for the first comparison; grow to 30. The models never see your requirements.
- Pick 2 to 4 models from 9. We preselect Claude Opus 5.5, GPT-6 Astra, Gemini 4 Carbon, and add the model you use today.
- Review and run. Before you pay you see the exact model versions, 5 independent attempts per case per model (5 cases × 3 models × 5 = 75 outputs), who receives your content, the price and a dated deadline.
- Read the report and choose a baseline. Which models meet your essential requirements, the important failures, cost per successful task (failed attempts and retries included) and measured end-to-end time. Inspect the outputs, then press Use this model as my baseline. A recommendation never changes it for you.
When a new model comes out
Open My benchmarks and press Test a new model. Your baseline is preloaded, comparable baseline results are reused, and the report shows where the new model improved or regressed, the cost and speed differences, and the evidence behind any switch recommendation. Add a case at any time: it starts a new version, and older reports stay under the version they ran on. See a sample “Should we switch?” report.
Pricing
US$299 one time
Automated first comparison
Choose a model on your own cases.
- Your cases (5 or more) × 2 to 4 models × 5 attempts each.
- Essential requirements, important failures, cost per successful task, end-to-end time, repeatability with denominators. Ties and “none qualified” are reported as such.
- Private report (web, HTML and PDF), and your benchmark saved for reruns if you choose.
- Dated deadline before you pay; automatic full refund if we miss it.
US$499 one time
Human-reviewed comparison
Find out why a model fails and test a fix.
Everything in the automated comparison, plus a reviewer who:
- investigates an important error,
- writes one revised prompt,
- runs a controlled rerun with the same models, cases and grading, and
- explains the before and after, including anything that got worse.
Request only for now: no charge to ask. Before any payment we confirm the reviewer, the deliverables and a delivery date in writing.
Testing a new model on a benchmark you already ran: quote on request, before anything is charged.
Questions
Why use ChatJudge instead of testing the models myself?
You can learn a lot by trying your prompt in a few chat windows. The harder question is whether you’d trust the same model to do the job again tomorrow. ChatJudge runs each task five times per model and shows how often it succeeds, how long it takes, and what each successful result costs, including retries. You get a private report you can share with your team when you decide which model to use.
Can I choose which models to compare?
Yes. We preselect three models to get you started. Keep those or choose any two to four from the nine available. If you already use a model in production, include it alongside a cheaper option or one you’re considering switching to. That way, the report answers the decision you actually face.
Who decides whether an answer is good?
A separate AI model grades the answers against the criteria for your task. None of the models you’re comparing grades its own work. Graders can disagree, too. In one sample comparison, one grader gave Claude Opus 5.5 a 77% pass rate and another gave it 90%. That 13-point gap is worth knowing before you rely on a single score.
What does “cost per successful task” mean?
We add up the model spend across all attempts, including retries, and divide it by the number of tasks completed successfully. If a model costs $1 across ten tasks and completes eight successfully, its cost per success is about $0.13. This lets you see whether a cheaper model still saves money once you account for failures.
What’s the difference between the US$299 and US$499 comparisons?
The US$299 automated comparison helps you choose a model based on quality, consistency, cost, and speed. With the US$499 reviewed comparison, a human reviewer investigates an important error or ambiguity, tests one revision to your prompt, and shows you the before-and-after results. Choose the reviewed option if you also want help improving how the task performs.
When will I get my report?
For the automated comparison, you’ll see the delivery deadline before you pay. We’ll email you a link to your private report. If we miss the deadline, we’ll automatically issue a full refund. For a reviewed comparison, we confirm the reviewer, scope, and delivery date before taking payment.
Is my content private?
Yes. Your cases, the models’ answers and your reports are encrypted when we store them, and we keep them out of our logs and analytics. When you create a benchmark you choose: keep it for reruns until you delete it (or 12 months after it was last used), or delete everything 30 days after each report. You can export or delete a benchmark at any time. Comparisons bought before benchmarks existed keep their 30-day deletion. The data controller is Advark Tech Pte Ltd. Questions go to hello@advark.ai.
Where these numbers come from. The 77% and 90% pass rates for Claude Opus 5.5 come from the sample comparison of six example tasks (6 synthetic tasks × 3 models × 5 runs, run 16 November 2026 at 12:49 SGT), graded by GPT-6 Sol and re-graded by Claude Sonnet 5.5. The US$1 across ten tasks example is illustrative arithmetic, not a measurement. A measured example: in the sample benchmark (5 synthetic contract cases × 3 models × 5 attempts, run 16 November 2026 at 18:45 SGT), Gemini 4 Carbon cost $0.011 per successful task across 25 attempts.
Extracting fields from documents?
The original document-extraction comparison is still available: three models on five of your redacted documents, scored field by field, US$299. Set up a document comparison · sample report (five synthetic invoices). Identifiers must match exactly, amounts must match in value and currency, dates are compared as dates, and ambiguous values are reported as ambiguous.
Privacy and deletion
- Who can open it: you, team members you share a benchmark with, and the ChatJudge operator (to run it). A named reviewer only if you buy a reviewed comparison and agree to it.
- Where your content goes: your cases go through the Codegraff gateway (operated by The Standard Harness Company of Singapore) to the models you pick, and to one grading model outside the ones you compare. Your requirements and example answers go only to the grader. Before each run we show this list and record your agreement, and we ask again before a new provider receives your content.
- Where it is kept: encrypted, in private storage on our server (Fly.io, Singapore).
- When it is deleted: your choice at creation: kept for reruns until you delete it (or 12 months after last use), or 30 days after each report. Orders bought before benchmarks existed are deleted 30 days after delivery, as promised. Unfinished document setups are deleted after 7 days.
- Your control: export a benchmark (cases, requirements, versions and results) and import it again; delete a report or a whole benchmark at any time.
- We keep case text, inputs, requirements and model outputs out of public pages, analytics and routine logs. Full details: privacy policy.
Terms
- First comparison: US$299, one time, your cases × the models you pick × 5 attempts, one private report. Automatic full refund if the complete report is not delivered by the deadline shown before payment. Results apply to the cases tested.
- Testing a new model later: priced by quote, agreed before anything runs or is charged.
- Human-reviewed comparison: US$499, one time, offered only after we confirm a reviewer, the deliverables and a delivery date. If the reviewed work is not delivered on time, that payment is refunded.
- Document-extraction comparison: US$299, three named models × five documents, same refund promise. Adding a reviewer to a delivered document comparison costs US$499 less what you already paid.
- Orders keep the terms they were bought under.
- Questions: hello@advark.ai. Advark Tech Pte Ltd, 71 Ayer Rajah Crescent, Singapore 139951.
