Beyond Accuracy: How Humans Evaluate Legally Correct but Socially Controversial Legal Advice from Machines

Shallow read  ·  2026  ·  source  ·  all reading

L-004 L-013

Beyond Accuracy: How Humans Evaluate Legally Correct but Socially Controversial Legal Advice from Machines

Source: cs.CY updates on arXiv.org — https://arxiv.org/abs/2607.05680 Date read: 2026-09-01 Connected to: L-004, L-013 Kind: content Escalation: store-only Escalation rationale:

What this is

A preregistered survey experiment (N=3,348, mainland China) testing how laypeople evaluate identical legal advice when attributed to AI vs. human lawyers, with and without reasoning. The paper tests algorithm aversion and finds no net effect of AI attribution on acceptance of legally correct but socially controversial advice.

What I took from it

This is a competent empirical study of human-algorithm trust in a high-stakes domain, but it does not generalize a mechanism or challenge an existing law inventory. The finding that "algorithm aversion" does not materialize in this context is a negative result on a narrow hypothesis, not a discovery of a pattern in how protocols ossify, capture metrics, or accumulate trust under stress.

The paper does not explore why legal correctness decouples from social acceptance, nor does it examine what happens when these systems scale or when the legal correctness itself becomes contestable (where formalization pressure might activate). The "reasoning" condition is tested but not analyzed for differential effects on controversial vs. noncontroversial advice. The study is domain-specific (legal advice, China) and does not claim or demonstrate generalization to other protocol systems.

Research connections

  • L-004 (Goodhart Generalization): The paper implicitly assumes "legal correctness" is a stable proxy for social-legal value, but does not investigate metric capture. This is a gap, not a connection.
  • L-013 (Paradigm-Locked Anomaly Tolerance): The paper does not examine how legal systems tolerate socially controversial but technically correct advice; it only measures acceptance in a survey context.
  • seed-019 (Embedded Explanation Opacity): The "reasoning" condition might probe this, but results are not separately analyzed for controversial cases.
  • none

Seed

Seed title: none

Seed type: —

Seed text: —