Two quotes land on your desk for the same launch review. One is for a benchmark run: a few hundred scored tasks and a dashboard. The other is for a red-team: two weeks of testers trying to break the system. They are not competing offers. They answer different questions, and confusing them is how teams end up with a reassuring score and an embarrassing launch.
What does a benchmark actually tell you?
A benchmark is a fixed set of tasks with a scoring rule. Run it on model A and model B, or on last week’s build and this week’s, and you get comparable numbers. That repeatability is the whole point.
The catch is that a benchmark only measures what someone thought to put in it. It samples expected behavior. If your support assistant answers nearly every billing question in the set correctly, that is useful, and it says nothing about whether a user can talk it into issuing refunds.
Public benchmarks have a second problem: their items end up in training data, so a high score may mean the model has seen the test. For product decisions, a private set built from your own traffic is worth more than any leaderboard.
What you should receive
- The item set itself, with reference answers or the grader code.
- Scores per item, not only an average.
- The exact model version, settings and date of the run.
What does a red-team actually tell you?
A red-team is adversarial. Testers start from the outcomes you fear (an exposed system prompt, a harmful instruction, a tool call that deletes data) and work backward to find inputs that produce them. They chain small weaknesses, change tactics when the model refuses and keep notes on what worked.
The output is a findings list, not a score. Each finding should carry the input, the output, the model version, a severity rating against your own policy and the steps to reproduce it. A good report also says what was tried and found nothing, because coverage is part of the evidence.
Vendors sell this in different shapes. Gray Swan AI describes Shade for red-team testing and an Arena network it says has more than 23,000 participants. Giskard sells continuous red-teaming on top of an open-source library. Toloka lists red-teaming alongside its evaluation work. The red-teaming services page lists the companies that describe this work on their own sites.
How do the two compare?
| Benchmark | Red-team | |
|---|---|---|
| Question | How often does it get known tasks right? | What can someone make it do wrong? |
| Output | A score you can track | A list of specific failures |
| Repeatable | Yes, by design | Partly: findings can be rerun, the search cannot |
| Who does it | Your engineers, or a vendor building private sets | Skilled testers, plus automated attack tools |
| When | Every meaningful change | Before launch, after fixes, at major releases |
| Main risk | Measures only what you thought of | Depends heavily on tester skill |
When should you buy which?
Buy a benchmark, or the tooling to run your own, when the question is comparative: which model to build on, whether a prompt change helped, whether a release regressed. You will run it many times, so the cost per run matters more than the cost of building it.
Buy a red-team when the question is about exposure: before a public launch, before giving an agent new tools, before a customer’s security review. You will run it a few times a year, so tester quality matters more than price per hour.
If your budget only covers one before a launch, ask which failure would hurt more: a slightly lower answer quality than a competitor, or a screenshot of your product doing something it never should. For most consumer-facing launches, that points to the red-team.
What should the contract say for each?
For a benchmark build, the contract should give you ownership of the items, rubrics and grader code, and promise the items are not reused for other clients. Reused items are how private sets turn into public ones.
For a red-team, the contract should define the harms in scope, the systems testers can reach, the number of rounds, how findings are reported, and the price of a retest once you have fixed things. It should also say testers work on staging with synthetic accounts, not on production data.
For both: a written answer on whether your prompts and outputs will train any model, and how long the vendor keeps them.
How do the two feed each other?
The best use of a red-team report is to turn every confirmed finding into a test case. Once an attack is written down, it stops being a red-team problem and becomes a regression problem: you want to know, on every build, that it still fails.
That loop is what turns two separate purchases into one evaluation program. The red-team searches for new failures; the benchmark makes sure old ones stay fixed. Over a year, the benchmark grows into a record of everything that has ever gone wrong, which is far more useful than a public leaderboard score.
Independent research groups work on the frontier version of this problem. METR, a nonprofit that works with AI developers on evaluations, says it has not accepted funding from AI companies. Its published work is good background on how hard adversarial testing of capable agents can be.
If you are comparing AI evaluation companies for a launch, ask each one which half of this loop it actually sells. Some are built around one half and sell the other as an add-on.
A score tells you how it usually behaves; a findings list tells you how it breaks. Budget for both before you ship.