Products / VZN Bench (Evals & Observability)

Partly liveWhat works today is listed first. What is still to come is listed under Not yet.

VZN Bench (Evals & Observability)

Evals and observability you can rerun. A deterministic evaluation harness: eight suites and 72 cases, run weekly, with reference answers that must pass.

Proposed pricing

The harness is free; hosted evals are metered per case.

Harness

R0 / one-time

Eight suites and 72 cases, run against reference answers.

  • Eight suites
  • 72 cases
  • Command-line runner

Hosted evals

Most common

Metered

Run the same cases on our hardware, priced per case.

  • Per-case metering
  • Weekly schedule
  • Results you can rerun

Enterprise

Talk to us

A published leaderboard and operator observability.

  • Leaderboard
  • Observability
  • Talk to us
Status

What runs today, and what is still open.

Works today

  • Eight suites: coding, writing, research, finance, marketing, service, safety, industry
  • A command line runner and a weekly workflow
  • A leaderboard command that ranks real run reports on the same cases, keeps the harness's own answers apart as controls, and refuses a report that does not add up

Not yet

  • A published leaderboard: it needs a real run for each model, and none has been run
  • Operator observability: the design is still open