The Benchmark Ceiling: Human Judgment, Evaluation Scarcity, and the Political Economy of AI Capability Measurement

The Benchmark Ceiling: Human Judgment, Evaluation Scarcity, and the Political Economy of AI Capability Measurement

Source: econ.GN updates on arXiv.org — https://arxiv.org/abs/2607.01254 Date read: 2026-09-01 Connected to: L-004, L-013 Kind: meta Escalation: store-only Escalation rationale:

What this is

An economics paper analyzing how AI capability benchmarks function as measurement instruments, arguing that as models saturate existing benchmarks, discriminating signal concentrates in expert-designed hard items, creating a structural scarcity of high-quality human judgment. The work addresses the political economy of evaluation infrastructure rather than proposing a sustained theoretical or empirical law about protocolized systems themselves.

What I took from it

This paper clarifies a methodological vulnerability in the research agenda: we are measuring capability and protocol behavior using instruments (benchmarks) that are themselves subject to metric capture and anomaly tolerance, but the paper does not theorize the protocol dynamics that emerge from this. It observes that elite judgment becomes a bottleneck, but does not model how this scarcity shapes optimization behavior, governance decisions, or the adoption of alternative evaluation schemes.

The work confirms the mechanism in L-004 (Goodhart Generalization) at the meta-level — benchmarks function as proxies for "true capability," and optimization pressure degrades their validity — but operates at the layer of how we measure systems, not how systems behave under measurement. It does not address whether capability measurement protocols themselves ossify, whether verification asymmetries emerge in audit vs. development, or whether governance systems develop anomaly tolerance toward benchmark saturation as an institutional problem. These are live questions for the new nature agenda, but this paper does not pursue them.

Research connections

• L-004: Confirms benchmark-as-proxy capture mechanism; does not model how optimizing agents respond when signal concentrates in hard items or how this reshapes evaluation protocol design.
• L-013: Tangentially relevant — suggests that measurement systems may tolerate anomalies (benchmark saturation, validity collapse) without triggering protocol revision, but does not provide a mechanism.
• seed-019: Related to embedded explanation opacity — if hard benchmark items require elite judgment, that judgment becomes opaque to scale; the paper does not explore this.

Method note

This paper illustrates a critical gap: research on protocolized systems should distinguish between (a) papers that analyze how protocols behave under stress, adoption, or optimization, and (b) papers that analyze the meta-infrastructure (benchmarks, evaluation, governance) through which we observe protocols. Both are necessary, but they operate at different levels. The benchmark scarcity argument is sociologically and economically sound but does not itself constitute a law of protocol behavior. To escalate such work, we would need to see how measurement scarcity shapes the evolution of the protocols being measured — i.e., how does the concentration of discriminating signal in hard expert-designed items change the structure of optimization targets, adoption incentives, or governance pressure on model developers? That argument is absent here.