Why AI Benchmark Coverage Misses Most American Work

AI benchmark coverage reaches only a narrow slice of work across the US economy, according to an analysis that estimates 90% of American working hours involve tasks with little or no coverage. That gap matters: benchmark scores can look precise while representing a limited—and uneven—sample of the work people actually do.

AI benchmark coverage leaves most occupations out

The analysis mapped prompts from 29 AI benchmarks to O*NET tasks, covering a total of 18,797 tasks across occupations. A task counted as covered only if it mapped to at least five benchmark prompts. By that measure, just 208 of 923 occupations had even one task represented, and six of 22 job families had no such occupations.

The 90% estimate is about working hours spent on tasks with little or no benchmark coverage, not a claim that AI cannot perform those tasks. That distinction is important. The analysis measures what benchmarks test; it does not establish what models can or cannot do across all work.

AI-exposed tasks and larger wage bills get more attention

Tasks rated as AI-exposed were about six times more likely to appear in benchmarks than tasks rated unexposed. Coverage also favored tasks in occupations with larger wage bills, even after accounting for AI exposure. The analysis describes smaller, middle-income knowledge-work jobs as undercovered.

My read: this is a problem of what gets measured before it is a verdict on model capability. If benchmark coverage tracks exposure and wage bills, then a benchmark suite may be more informative about the work researchers and evaluators prioritize than about the full range of work AI might encounter. That is an interpretation of the coverage pattern, not a finding that the benchmarks are useless or that model performance is poor on omitted tasks.

What benchmark gaps mean for evaluation

A benchmark result is only as useful as the tasks it represents. The analysis gives a concrete reason to be cautious when using current benchmark scores to make broad claims about AI’s readiness for work: most American working hours, by its estimate, concern tasks with little or no representation. It does not show how results would change with a more representative benchmark set, or whether the five-prompt threshold is the right bar for meaningful coverage.

The authors call the findings provisional because the initial benchmark list is incomplete and its selection criteria involve subjective judgment. That caveat is not a footnote to ignore; it limits how firmly we should treat the exact coverage figures. But it does not erase the practical concern. Better coverage would make comparisons more relevant, while still leaving the separate question of how well models perform on those tasks.

For now, I would treat benchmark scores as evidence about tested tasks, not as a proxy for the whole labor market. The next useful signal is whether broader benchmark inventories change this coverage picture—especially for the smaller, middle-income knowledge-work jobs the analysis says are underrepresented.