Smart Enough to Go Extinct? An Evolutionary Challenge to the Value of General Intelligence and Its Ethical Implications for AGI

Smart Enough to Go Extinct? An Evolutionary Challenge to the Value of General Intelligence and Its Ethical Implications for AGI

Source: cs.CY updates on arXiv.org — https://arxiv.org/abs/2608.10730 Date read: 2026-09-02 Connected to: L-004, seed-045 Kind: content Escalation: store-only

What this is

A philosophical and evolutionary argument that challenges the assumed adaptive value of general intelligence across evolutionary timescales, using examples of extinct high-intelligence species (horses, humans in certain contexts) to argue that domain-general cognition may be maladaptive or neutral under many selective regimes. The paper extends this skepticism to AGI development, questioning whether optimization for general intelligence is itself a sound objective.

What I took from it

The paper operates at a level of abstraction too distant from protocol dynamics to generate actionable constraint on the laws under accumulation. Its core claim—that general intelligence lacks demonstrated adaptive value over evolutionary time—is a claim about biology and evolutionary fitness, not about the structural dynamics of formalized, rule-bound systems.

However, there is a latent connection to L-004 (Goodhart Generalization) and L-005 (Gall Generalization): the paper implicitly identifies a case where optimization toward a proxy metric (general intelligence capacity) may produce value collapse or extinct lineages. But this operates at the wrong scale—it is paleontological, not mechanistic in the protocol sense. The paper does not examine how a system that optimizes for legible general-intelligence metrics behaves under enforcement pressure, nor does it trace how that optimization reshapes the protocol substrate itself.

The ethical framing is consequentialist but not structural. There is no mechanism analysis of how AGI optimization protocols might ossify, capture their own objectives, or produce coordination failures—the stuff of the new-nature laws.

Research connections

• L-004: Goodhart Generalization applies if "general intelligence" is treated as a measurable proxy for unspecified long-term value; the paper's evolutionary evidence suggests such proxy collapse occurs, but at a timescale and causal distance that does not clarify the mechanism in protocol systems.
• L-005: Gall Generalization is tangentially relevant—the paper suggests that radical redesign of intelligence from first principles may fail where incremental evolution succeeded—but this is applied to biological evolution, not protocol restructuring.
• seed-045: The paper connects to value-collapse dynamics under selection pressure, but does not examine how such pressure is operationalized in formal systems with legible targets.

Seed

Seed title: none

Seed type: —

Seed text: —

---

Store rationale: This is a well-reasoned philosophical contribution to AGI ethics, but it is not a primary source on protocolized systems. It does not present a sustained empirical or theoretical argument about laws of formalized coordination. It restates L-004 (Goodhart) at the biological level without mechanism evidence specific to protocol dynamics. No new law-shaped fragment emerges that does not already inhabit the seed pool or active law inventory. File and track for cross-domain pattern later if similar dynamics appear in formal system studies.