The Measurement Revolution? Credible Measurement and Inference in the Age of AI

Source: econ.GN updates on arXiv.org — https://arxiv.org/abs/2608.23524 Date read: 2026-09-02 Connected to: L-004, seed-026 Kind: content Escalation: escalate-to-deep Escalation rationale: This is a primary source identifying a sustained structural shift in measurement legibility that directly extends L-004 (Goodhart Generalization) and opens a new mechanism: when AI makes previously unmeasurable constructs measurable at scale, the bottleneck shifts from scarcity to proliferation of incommensurable proxies, which is a distinct failure mode from metric capture under optimization pressure.

What this is

A review paper arguing that AI's capacity to convert unstructured data into structured variables at scale has inverted the economics of measurement in empirical sciences. The bottleneck is no longer finding a scalable measure but selecting among many plausible ones that may support contradictory conclusions—creating new inference credibility problems distinct from measurement cost.

What I took from it

This work identifies a regime shift in how Goodhart-type failure modes manifest. L-004 assumes a single proxy under optimization pressure; this paper shows that AI-driven legibility creates multiplicity rather than scarcity. The critical insight: when a previously immeasurable construct (e.g., "economic sentiment," "firm innovation," "neighborhood quality") becomes measurable via multiple AI extraction pipelines, no single proxy is privileged, and different measurement choices yield genuinely different causal inferences. This is not metric capture—it's metric underdetermination under legibility abundance.

This has immediate implications for protocol design: protocols that depend on measurement consensus (lending protocols using AI-extracted credit signals, allocation protocols using AI-measured need, governance protocols using AI-legible "transparency" signals) face a new failure mode: structural disagreement about what the measure means, even when all parties agree on the measure's accuracy. The formalization that was supposed to resolve disputes now multiplies them.

Research connections

L-004: Extends rather than challenges—Goodhart applies to single-proxy optimization; this identifies when legibility abundance prevents proxy stabilization, leaving unmeasurable constructs fragmented across competing formalizations.
L-003 (Formalization Ratchet): Directly relevant—under scaling/coordination stress, informal measures get formalized, but AI-driven legibility means formalization doesn't converge to one formal measure; it proliferates.
seed-062 (Formalization Opacity Collapse): The automation pipeline that makes measurement cheap makes the choice among measures opaque—opacity moves from the measurement process to the selection criteria.
seed-073 (Correlated Failure Under Proxy Consensus): When multiple AI-extracted proxies correlate spuriously, systems built on different proxies may fail in lockstep despite appearing independent.
seed-080 (Proxy Collapse Under Upstream Asymmetry): Relevant if upstream AI models are trained asymmetrically (e.g., on privileged data), creating hidden non-equivalence among "equivalent-looking" proxies.

Seed

Seed title: Legibility Multiplication as Protocol Failure Mode

Seed type: insight

Seed text: When computational legibility makes a previously unmeasurable construct measurable via multiple algorithmic pathways, protocol systems that depend on measurement consensus do not stabilize on a single proxy—they fragment across incommensurable formalizations. Each formalization is individually defensible; no measurement error explains divergence. This differs from Goodhart capture (single proxy, optimization pressure) and creates a failure mode where shared measurement infrastructure paradoxically enables divergent causal inference and decision outcomes. This risk is highest in safety-critical or high-stakes protocols where measurement legibility is treated as equivalent to operational legitimacy.