Vol. 1 · Claim → Evidence → Challenge → Judgment

Prove the benchmark, not just the score.

AI labs publish numbers. BenchProof asks whether the evidence fairly supports the sentence those numbers are wrapped in.

Verify a claim Explore claims

Claims: 6
Open: 0
Finalized: 6
Challenges: 5

Live Studionet verdict · not a mock

Claim 1 · INSUFFICIENT_EVIDENCE

Validators reached MAJORITY_AGREE. A harness URI without raw results is not enough to support a published score.

Open the case file Evaluation tx

The problem

Benchmark claims are easy to publish and difficult to independently verify.

A leaderboard compares two numbers. BenchProof compares the experiment. If Model A had five retries and Model B had one, “20% better” is a marketing sentence, not a result.

Exhibit 01

Old baseline

Compare against a 2023 checkpoint and call it “GPT-4” in 2026. The number looks large. The comparison is not.

Exhibit 02

Cherry-picked tasks

Drop the instances the model failed, keep the ones it passed, then publish a headline accuracy.

Exhibit 03

Different retries

Five samples with majority vote versus one greedy decode. Same benchmark name, different experiment.

Exhibit 04

Different prompts

One model gets a planning scratchpad and tools. The baseline gets a bare completion prompt.

Exhibit 05

Hidden failed runs

Timeouts and empty patches silently removed from the denominator.

Exhibit 06

Misleading headline

A 3-point gain on a matched split becomes “20% better at agentic coding.”

How it works

Five steps. One question.

01
Submit claim
Exact statement, models, versions, metric, and methodology.
02
Attach evidence
Hashes and references live on-chain. Large artifacts stay off-chain.
03
Challenge
Anyone can contest retries, prompts, baselines, leakage, or headlines.
04
GenLayer evaluates
Validators reach consensus on whether the evidence fairly supports the claim.
05
Verdict is verifiable
SUPPORTED is not a score. It is a public, inspectable judgment.

Example case

“20% better” with unequal retries.

A published agentic-coding claim reported a 20% relative gain. The evidence showed Model A received five retries and extra tools; Model B received one attempt. The headline did not disclose the gap. BenchProof does not average the two scores. It asks whether the sentence is fair.

Open the case file

Misleading

Case B · Finalized · MISLEADING

Model A is 20% better than Model B on agentic coding tasks.

Retries: 5 vs 1
Tools: web+python vs bash
Headline discloses gap: no

Why GenLayer

This cannot be reduced to if scoreA > scoreB.

Whether a claim is misleading is a judgment over unstructured evidence: reports, configs, hashes, challenges. Intelligent Contracts let validators reason over that record, then agree on meaning rather than on a byte. Remove GenLayer and BenchProof is just another form. With it, the verdict is a consensus artifact.