Eörs Szathmáry

Müller, Steels, and Szathmáry made this timely case that evolvable AI (eAI) should be treated as a distinct class of risk because essential elements of life such as replication, heredity, variation, and selection were also created in digital systems (1). Their distinction between human-controlled “breeder” settings and open “ecosystem” settings is especially useful. However, this distinction will remain difficult to govern unless it is translated into measurable thresholds.

Abstract Evolvable AI (eAI), i.e., AI systems whose components, learning rules, and deployment conditions can themselves undergo Darwinian evolution, may soon emerge from current trends in generative, agentic, and embodied AI. We argue that this possibility has been underappreciated in debates on AI safety and existential risk. Here, we ask under what technical and ecological conditions AI becomes evolvable, what kinds of behaviors are then likely to emerge, and how such systems could be governed.

Abstract Building on the algorithmic equivalence between finite population replicator dynamics and particle filtering based approximation of Bayesian inference, we design a computational model to demonstrate the emergence of Darwinian evolution over representational units when collectives of units are selected to infer statistics of high-dimensional combinatorial environments.