Products / VZN Bench (Evals & Observability)
Partly liveWhat works today is listed first. What is still to come is listed under Not yet.
VZN Bench (Evals & Observability)
Evals and observability you can rerun. A deterministic evaluation harness: eight suites and 72 cases, run weekly, with reference answers that must pass.
Proposed pricing
The harness is free; hosted evals are metered per case.
Harness
R0 / one-time
Eight suites and 72 cases, run against reference answers.
- Eight suites
- 72 cases
- Command-line runner
Hosted evals
Most commonMetered
Run the same cases on our hardware, priced per case.
- Per-case metering
- Weekly schedule
- Results you can rerun
Enterprise
Talk to us
A published leaderboard and operator observability.
- Leaderboard
- Observability
- Talk to us
Status
What runs today, and what is still open.
Works today
- Eight suites: coding, writing, research, finance, marketing, service, safety, industry
- A command line runner and a weekly workflow
- A leaderboard command that ranks real run reports on the same cases, keeps the harness's own answers apart as controls, and refuses a report that does not add up
Not yet
- A published leaderboard: it needs a real run for each model, and none has been run
- Operator observability: the design is still open