The Limits of LLM Forecasting: Parametric Knowledge Gaps Across Conflict Zones

Source: cs.CY updates on arXiv.org — https://arxiv.org/abs/2607.00018 Date read: 2026-09-01 Connected to: L-004, L-013 Kind: content Escalation: store-only Escalation rationale:

What this is

An empirical evaluation of LLM conflict forecasting across asymmetrically covered conflict zones, documenting a 224× media coverage gap and showing that LLMs fail to forecast escalation — they categorize it based on training distribution. The work uses a 660-case held-out test set comparing Llama-3.3-70B and GPT-4o against structured baselines.

What I took from it

The paper demonstrates a failure mode of metric proxy collapse under information asymmetry. LLMs trained on English-language media corpora inherit the coverage asymmetry as a parametric fact: they cannot forecast what they have not seen enough signal about. The result is not graceful degradation but qualitative misclassification — the system switches to categorizing conflicts by learned patterns (e.g., geopolitical clusters, historical precedent) rather than forecasting dynamics.

This directly confirms L-004 (Goodhart Generalization): the proxy — media coverage as a signal of conflict severity or escalation risk — is optimized under training pressure, but the proxy itself is corrupted upstream by editorial and institutional asymmetry. The system learns to predict coverage, not conflict. It also supports L-013 (Paradigm-Locked Anomaly Tolerance): the anomaly (systematic failure on under-covered zones) is tolerated because the evaluation framework itself inherits the same asymmetry. If test sets are drawn from the same media corpora, the system appears calibrated when it is merely memorizing bias.

Research connections

L-004: Media coverage asymmetry is a corrupted proxy; optimization under training pressure causes LLMs to forecast the proxy (coverage patterns) rather than the ground truth (escalation dynamics).
L-013: Evaluation frameworks that inherit the same parametric bias as the training corpora will tolerate systematic failures on under-represented domains as "acceptable variance."
seed-019 (embedded-explanation-opacity): LLM forecasting outputs lack causal grounding; they produce confidence scores that appear legitimate but reflect training distribution, not predictive competence.
seed-045 (intelligence-entropy-monotonic-disorder): Parametric knowledge gaps are not random noise — they are structured by upstream institutional asymmetry and are monotonically resistant to scaling.

Seed

Seed title: Proxy Collapse Under Upstream Asymmetry in Automated Systems

Seed type: observation

Seed text: When a predictive or forecasting system is trained on a proxy that is itself asymmetrically distributed (e.g., media coverage, institutional audit trails, labeled datasets), the system does not learn a degraded version of the target task — it learns to optimize the proxy distribution itself. Under sufficient parametric pressure, the system's outputs become a direct function of proxy density rather than ground truth. This is distinct from L-004 (Goodhart capture) because the metric is corrupted before optimization begins. The failure mode is not convergence to a wrong target; it is structural misalignment between what the system is trained to predict and what it appears to predict. This should generalize across any automated system relying on institutionally filtered or editorially asymmetric training signals.