AI Benchmark Index
AI Benchmark Index catalogs 0 AI benchmarks with source-linked facts about data, grading, and reported scores.
Guides
- How to choose an AI benchmark
Choose an AI benchmark by defining the decision first, then checking task fit, grader validity, data access, saturation, and provenance.
- How to run an AI benchmark yourself for under $20
Keep an AI benchmark run under $20 by fixing the question, sampling tasks, estimating tokens, using deterministic graders, and saving a reproducible run record.
- Silly benchmarks that tell you something real (pelican, skateboard, and friends)
Playful tasks such as drawing a pelican on a bicycle can expose planning, geometry, tool use, and persistence when their grader makes the signal explicit.
- What a saturated benchmark is and how to spot one
A saturated benchmark no longer separates strong systems well; spot it by score clustering, ceiling effects, stale tasks, and shrinking uncertainty-aware gaps.
- Where benchmark data comes from and what its licence lets you do
Benchmark data may be written, sampled, generated, or drawn from public datasets; its licence determines whether you may download, modify, and redistribute it.
Published by AI Benchmark Index. How we check each benchmark.