AI evaluation vendors and tools
20 companies sell AI evaluation in some form. The software vendors publish prices, from $0 starter tiers to Braintrust's $249 a month Pro plan; the managed services that supply raters and red-teamers quote per project and publish nothing.
We are independent of every firm below: none owns, sponsors or endorses this site. Nothing here comes from our own testing or audits; each row reflects what the company itself publishes.
Last verified 7 November 2026, against each company's own page.
Which eval platforms publish their prices?
Software your engineers run: test sets, scorers, model graders, traces and CI gates. Prices are per seat, per plan or both.
| Company | What it sells for evaluation | Delivery | Published pricing | Security claims on its own page |
|---|---|---|---|---|
| Braintrust | Runs evals, scores outputs and traces production LLM apps | Software you run | Starter $0 a month (1 GB processed data, 10k scores, 14-day retention); Pro $249 a month (5 GB, 50k scores, 30-day retention); Enterprise custom | SOC 2 Type II (stated on pricing page) |
| LangSmith (LangChain) | Tracing, testing and monitoring for LLM agents | Software you run | Developer: free per seat, 5k base traces monthly. Plus: $39 per seat monthly, 10k base traces, then pay as you go. Enterprise: custom | Not published |
| Confident AI (DeepEval) | Cloud platform built on DeepEval, its Apache-2.0 open-source eval framework for Python and TypeScript | Software you run | Free tier; paid plan prices not shown on the page read | SOC 2 Type II (self-stated) |
| Giskard | Agent testing and continuous red-teaming, with an open-source Python library | Software you run | Open-source edition free (runs locally); Professional plan price not shown | Not published |
| GalileoOwner: Cisco (acquisition completed May 2026) | Evaluation, observability and guardrails for AI agents | Software you run | Not published | Not published |
| HumanSignal (Label Studio) | Label Studio for human review and scoring of model outputs; open-source Community Edition | Software you run | Starter Cloud: $99 monthly covers 12 users, each extra user $49 monthly. Enterprise: custom | SOC 2 stated for the Enterprise plan |
Disclosure: vendors in this table can pay us a fee when an introduction we made ends in a signed deal. The listing, its order and its wording are never for sale.
Which companies sell a platform and evaluation services?
These run their own software and can also staff the work. Pricing is usually a subscription or usage units for the software and a quote for the people.
| Company | What it sells for evaluation | Delivery | Published pricing | Security claims on its own page |
|---|---|---|---|---|
| Kili Technology | Annotation, evaluation and post-training data platform, with optional specialist annotators | Platform and services | Free trial $0 (2 to 5 seats, 100-asset limit); Grow and Enterprise custom | SOC 2 Type II, ISO 27001 (pricing page) |
| Labelbox | Labeling platform billed in Labelbox Units (LBUs), with labeling services as an add-on | Platform and services | 20 LBU per data row for live multimodal and LLM work; 500 free LBU a month on free accounts; no dollar price per LBU published | Not published |
| SuperAnnotate | Platform for building annotation, evaluation and fine-tuning datasets | Platform and services | Starter, Pro and Enterprise plans listed; no prices shown | Not published |
| Scale AI | Data engine for training data, plus model evaluation | Platform and services | Not published | Not published |
| Snorkel AI | Expert datasets, benchmarks and evaluations for AI labs and enterprises | Platform and services | Not published | Not published |
| Gray Swan AI | Shade for red-team testing, Cygnal for input and output filtering, and Arena, a red-teaming network it says has 23,000+ participants | Platform and services | Not published | Not published |
| Centific | AI Data Foundry: preference data, SFT, RL environments and safety evaluation | Platform and services | Not published | Not published |
Disclosure: vendors in this table can pay us a fee when an introduction we made ends in a signed deal. The listing, its order and its wording are never for sale.
Which managed services run human evaluation for you?
People-first vendors: expert networks, rating workforces and a participant marketplace. Only Prolific publishes a rate, and only for the participants' share.
| Company | What it sells for evaluation | Delivery | Published pricing | Security claims on its own page |
|---|---|---|---|---|
| Surge AI | Human evaluation, rubrics and verifiers, RLHF and SFT data; says it works in over 70 languages | Managed service | Not published | Not published |
| Toloka | Auto-verifiable evaluation tasks, preference data and red-teaming; based in Amsterdam | Managed service | Not published | Not published |
| TELUS Digital | GenAI post-training data and red-teaming; says it has a 1M+ contributor community | Managed service | Not published | Not published |
| Sama | Managed annotation and validation, mostly computer vision and generative AI data | Managed service | Not published | ISO 9001, ISO 27001, TISAX (security page) |
| Mercor | Expert network supplying human evaluation and the APEX benchmarks to AI labs | Managed service | Not published | Not published |
| Turing | Post-training data and evaluation for AI labs, including RL environments | Managed service | Not published | Not published |
| Prolific | Recruits paid participants for research and human data, including human ratings of AI output | Participant marketplace | Participant minimum $8 an hour, recommended $12 an hour; platform fee charged on top (rate not stated on the page read) | Not published |
Disclosure: vendors in this table can pay us a fee when an introduction we made ends in a signed deal. The listing, its order and its wording are never for sale.
Who is missing from this list, and why?
Some names come up in every eval conversation but are not vendors you can hire today, or are not separate companies.
- Haize Labs. The red-teaming startup was acquired by Beacon, a Toronto software holding company, in September 2026, and it is not clear it still takes outside clients.
- Alignerr. A worker-side brand of Labelbox. Buy through Labelbox.
- METR. A research nonprofit that works with AI developers on evaluations of frontier agents. It is a source, not a vendor, and its about page says: "METR has not accepted funding from AI companies."
- Epoch AI. A research organization that publishes data on AI trends. Useful reading, not a supplier.
What changed in vendors' published terms?
Pricing moves, takeovers and rewritten terms among the firms above, latest at the top.
- Baseline read of each company page behind the tables above; plan prices and limits shown are as they stood that day.
- Snorkel AI announced a $350M Series E at a $3.5B valuation. Source: Snorkel AI press release.
- Beacon, a Toronto software holding company, announced it had acquired the red-teaming startup Haize Labs. We no longer list Haize Labs as an available vendor. Source: The Next Web.
- Prolific dated its payment model article: participant minimum $8 an hour, recommended $12 an hour, platform fee on top. Source: Prolific help center.
- Cisco said its acquisition of Galileo had completed (announced 9 April 2026). Galileo is listed as Cisco-owned. Source: Cisco blog.
Questions teams ask
Which AI evaluation vendors have a free tier?
From the pages we read: Braintrust (Starter, $0), LangSmith (Developer, $0 a seat), Confident AI (a free tier), Kili Technology (a free trial capped at 100 assets), Labelbox (500 free LBU a month) and open-source editions from Giskard and HumanSignal (Label Studio).
Why do so many say "Not published"?
Managed evaluation is priced per project, so firms such as Scale AI, Surge AI, Toloka and TELUS Digital give quotes rather than rate cards. Where a page did not state a figure, we say so instead of guessing.
Is a SOC 2 badge on a vendor page enough?
No. It tells you what to ask for. Request the report itself under NDA and read its date, type (Type I or Type II) and which systems it covers. Sama, for example, lists ISO 9001, ISO 27001 and TISAX on its security page.
Who owns Galileo now?
Cisco. It announced the deal on 9 April 2026 and said on 22 May 2026 that the acquisition had completed.
Are you affiliated with any of these companies?
No. None of them owns, runs or endorses this site. A vendor can pay us after an introduction that becomes paid work; the how this works page explains it.
Pages that pick up from here
- Types of AI evaluation which kind each vendor actually covers
- Red-teaming services the adversarial side, vendor by vendor
- Eval budget turn the published prices into a first-round figure
Where these facts come from
- braintrust.dev: Braintrust pricing page; as seen on 7 November 2026
- langchain.com: LangChain pricing page; as seen on 7 November 2026
- confident-ai.com: Confident AI DeepEval page; as seen on 7 November 2026
- giskard.ai: Giskard pricing page; as seen on 7 November 2026
- blogs.cisco.com: Cisco blog on the Galileo acquisition; as seen on 7 November 2026
- humansignal.com: HumanSignal pricing page; as seen on 7 November 2026
- kili-technology.com: Kili Technology pricing page; as seen on 7 November 2026
- docs.labelbox.com: Labelbox billing docs; as seen on 7 November 2026
- superannotate.com: SuperAnnotate pricing page; as seen on 7 November 2026
- researcher-help.prolific.com: Prolific payment model article; as seen on 7 November 2026
- scale.com: Scale AI data engine page; as seen on 7 November 2026
- surgehq.ai: Surge AI products page; as seen on 7 November 2026
- snorkel.ai: Snorkel AI press release, 22 Sept 2026; as seen on 7 November 2026
- toloka.ai: Toloka about page; as seen on 7 November 2026
- telusdigital.com: TELUS Digital data for AI training page; as seen on 7 November 2026
- grayswan.ai: Gray Swan AI about page; as seen on 7 November 2026
- sama.com: Sama data security and trust page; as seen on 7 November 2026
- centific.com: Centific AI Data Foundry page; as seen on 7 November 2026
- mercor.com: Mercor homepage; as seen on 7 November 2026
- turing.com: Turing RL environments page; as seen on 7 November 2026
- thenextweb.com: The Next Web, Beacon acquires Haize Labs; 17 Sept 2026
- metr.org: METR about page; as seen on 7 November 2026
Get a quote
Tell us what you need evaluated and which of these you are already talking to. We'll say which vendors' published offers fit the brief.
We are an introduction service, not an evaluation vendor, and we may be paid by the company we introduce you to.
Please do not paste prompts, logs, model outputs or any other data here. Describe the project in words; data only ever moves under a contract you sign with the vendor.
Brief received.
Here is what happens next:
- We read the brief and check it against the vendors' published offers and limits.
- We email you the names of the vendors that fit and why, with links to their own pages.
- Reply yes to the ones you want to hear from. Anyone you do not pick never sees your details.