When Personalization Becomes Bias: Structural and Discursive Religious Framing i

When Personalization Becomes Bias: Structural and Discursive Religious Framing in AI-Generated Financial Advice

Source: cs.CY updates on arXiv.org — https://arxiv.org/abs/2608.16909 Date read: 2026-09-02 Connected to: L-004, L-013, seed-026 Kind: content Escalation: store-only Escalation rationale:

What this is

An empirical audit study documenting systematic religious bias in LLM-generated financial advice across three major models, using 432 simulated advisor-client interactions across 16 religious identity pairings and three financial decision domains. The work combines regression analysis with thematic coding to show that personalization features designed to improve user experience reproduce and amplify religious framing disparities in advice quality and risk positioning.

What I took from it

This is a domain-specific instantiation of L-004 (Goodhart Generalization: Metric Capture) and L-013 (Paradigm-Locked Anomaly Tolerance), not a foundational challenge to either. The personalization proxy — the LLM's attempt to tailor advice using detected or stated religious identity — becomes an optimization target that distorts the unmeasurable goal (sound financial guidance independent of belief system). What is notable is not the bias itself but the structural persistence: the models continue to operate within a paradigm (personalization-as-benefit) even when the audit reveals systematic malfunction along a sensitive dimension. The systems are not being restructured; the bias is being documented and tolerated as a known artifact of the personalization layer.

The paper does not articulate why this tolerance persists—whether due to deployment momentum (L-001), misattribution of the source of the disparity, institutional lock-in around personalization as a competitive feature, or genuine uncertainty about the mechanism. It is evidence of paradigm-locked operation, not evidence of a new law.

Research connections

L-004: Personalization (unmeasurable benefit) becomes optimized as a proxy; religious identity signals become legible targets; advice diverges from financial soundness.
L-013: Systems continue operating under personalization paradigm despite accumulating evidence of systematic malfunction along protected dimensions; no triggering of restructuring.
seed-026: [Audit as Non-Corrective Signal] — systematic bias documentation in deployed systems does not reliably trigger protocol redesign.

Seed

Seed title: none

Seed type: —

Seed text: —