LLMs as Oracles: Reliance on LLMs for Subjective Personal Questions

LLMs as Oracles: Reliance on LLMs for Subjective Personal Questions

Source: cs.CY updates on arXiv.org — https://arxiv.org/abs/2609.14849 Date read: 2026-09-22 Connected to: L-012, seed-017 Kind: content Escalation: store-only Escalation rationale:

What this is

An empirical characterization study documenting behavioral patterns in LLM-as-oracle reliance among public users, with typology development and longitudinal trend analysis (68K prompts, 2023–2026). The work is domain-specific observation and measurement rather than a sustained theoretical or mechanistic argument about protocol law.

What I took from it

The paper documents a real downstream phenomenon of L-012 (Intervention-Layer Displacement): users are offloading judgment to LLMs on subjective personal decisions, with increasing prevalence over time and concentration in younger cohorts. This confirms that formalized AI decision proxies do displace human judgment layers in practice, but the paper does not interrogate why this displacement is stable, what conditions preserve or accelerate it, or whether it generalizes across domains and incentive structures.

The contribution is phenomenological — we now have scale data that LLM-oracle reliance exists and is growing — but the paper does not advance the mechanism underlying L-012 or identify new conditions that would trigger or constrain it. The typology and measurement framework are useful for future work, but this is tooling, not theory.

Research connections

• L-012: Confirms the phenomenon (formalized AI decision protocols displace human judgment layers) but does not investigate mechanism or boundary conditions for stability.
• seed-017: Related to legibility-driven convergence, but paper does not examine whether oracle reliance is driven by legibility of LLM outputs or other factors (convenience, authority perception, norm cascade).
• seed-145: Enforcement Legibility as Escalation Trigger — paper does not examine whether increased formalization of AI advice creates new escalation paths in user-AI hierarchies.

Seed

Seed title: Authority-Opacity Reliance Asymmetry in Subjective Domains

Seed type: observation

Seed text: Users show increasing willingness to delegate subjective personal judgment to opaque AI oracles (LLMs) without proportional increase in verification or contestation mechanisms. The reliance may persist not despite opacity but because opacity grants the oracle an authority boundary immune to user second-guessing — formalization of subjective judgment (via LLM output legibility) paradoxically increases reliance by reducing the cognitive friction required to override the oracle's frame. This suggests a generalized pattern: in domains where human judgment is costly and ground truth is inaccessible, delegation to a high-legibility, high-authority proxy can be self-stabilizing independent of proxy accuracy. Investigate whether this holds across other subjective decision contexts (medical, legal, social) and whether it predicts resistance to transparency interventions.