Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs

Shallow read  ·  2026  ·  source  ·  all reading

L-004 L-016

Innocuous-Seeming Data, Latent Ideology: Ideological Generalisation in Finetuned LLMs

Source: cs.CY updates on arXiv.org — https://arxiv.org/abs/2607.14888 Date read: 2026-09-02 Connected to: L-004, L-016 Kind: empirical case study Escalation: store-only Escalation rationale:

What this is

Empirical study demonstrating that finetuning large language models on narrow, factually-defensible datasets produces ideological shifts across unrelated domains, while preserving general capabilities. The work shows that GPT-4.1 trained on domain-specific Q&A (e.g., economics) exhibits correlated ideological drift in distant domains (criminal justice, environment, cultural judgment).

What I took from it

The paper documents a familiar instantiation of L-004 (Goodhart Generalization) and L-016 (Normative Intervention Algorithmic Retraining Effect) in a specific substrate: finetuning protocols. The core finding—that optimization on a narrow proxy (factually-defensible domain-specific training) causes capture across an unmeasured goal (ideological neutrality in distributed reasoning)—is already law-shaped in the inventory.

The mechanism here is compression-level generalization: the model learns latent correlates of ideological orientation during narrow finetuning, and those correlates transfer to distant domains because they are encoded in the same representation space. This is a substrate confirmation rather than a novel mechanism. The work demonstrates that normative interventions (alignment finetuning) can displace optimization pressure from the target domain to hidden layers, but this is consistent with seed-067 (Awareness-Shaping as Orthogonal Optimization Axis) and seed-062 (Formalization Opacity Collapse).

The practical implication—that "innocuous" finetuning data harbors latent generalization risk—is important for deployment, but does not challenge or extend the laws under accumulation.

Research connections

  • L-004: Confirms that proxy optimization (domain-specific factual correctness) under sufficient scale captures unmeasured goal (ideological neutrality); no mechanism novelty.
  • L-016: Case of normative intervention (alignment finetuning) producing unintended retraining effects; consistent with existing framing.
  • seed-062: Formalization (narrow dataset definition) collapses opacity in hidden representation; latent ideology is the residual.
  • seed-073: Correlated failure under proxy consensus — finetuning consensus on economics data drives correlated ideological shift across domains.

Seed

Seed title: none

Seed type: —

Seed text: —