Anthropic urges ‘pause’ or ‘slowdown’ of AI development after Leo’s encyclical
Anthropic co‑founders Jack Clark and Marina Favaro warned that AI systems could soon autonomously design successors, risking loss of human control. Their warning follows Pope Leo XIV’s encyclical “Magnifica Humanitas,” which cautions against building a “new Tower of Babel” in AI. The company calls for a global pause or slowdown of frontier AI development to allow society to address safety, alignment and ethical issues. Anthropic plans to engage policymakers, researchers and the public, and will publish a follow‑up document based on those discussions.
4 months ago
Anthropic, the AI firm behind Claude, has joined Pope Leo XIV’s call for greater caution on artificial intelligence by urging a global pause or slowdown in frontier AI development. The company’s leaders warn that recursive self‑improvement could let AI systems design their own successors, raising the risk of losing human control. Their appeal echoes the new papal encyclical Magnifica Humanitas, which likens unchecked AI progress to a “new Tower of Babel” and urges the “disarming” of technology so it does not dominate humanity. Anthropic is seeking dialogue with policymakers, researchers, and religious leaders to shape ethical safeguards and promote international cooperation 1 2.
Assess Catholic ethical guidance on AI’s autonomous development
Catholic ethical guidance on the autonomous development of AI centers on one controlling principle: human persons—by conscience and freedom—must remain the moral agents who decide how (and whether) AI is used, even when systems appear to “choose” or improve themselves.
The Church’s approach is not anti-technology; it is anthropological—rooted in what it means to be human. Pope Leo XIV frames moral discernment about AI as safeguarding “the primacy of the human person,” so that “it will always be human intelligence, with its conscience and freedom, that guides technical innovations and responsibly determines their use and limits.”
This directly challenges the ethical temptation to treat machine self-modification or independent output as if it were moral autonomy. The Magisterium-related Note from the Dicasteries likewise stresses an integral vision: human intelligence (in God’s image) must express itself through responsible stewardship, not through surrender of governance to “functional automatisms.”
In Pope Francis’s address to the G7 on AI, the ethical boundary is expressed in unambiguous terms:
“We need to ensure and safeguard a space for proper human control over the choices made by artificial intelligence programs: human dignity itself depends on it.”
And more sharply:
“No machine should ever choose to take the life of a human being.”
The same logic is applied beyond weapons. Francis warns that even sophisticated systems can create a “risk” of de facto delegating the last word concerning a person’s future—for example in criminal justice decisions supported by predictive AI.
He explains why this is ethically serious: because machine classifications can “implicitly incorporate prejudices” embedded in data categories (e.g., ethnicity, prior minor offenses), while “human beings are always developing” in ways a machine cannot fully grasp.
In Catholic terms, the issue is not that AI lacks sophistication, but that it lacks moral agency: it does not possess conscience, responsibility, or the lived interiority necessary for moral judgment.
Catholic ethical guidance becomes most concrete in the context of lethal force. Pope Francis calls for an urgent reconsideration—and ultimately a ban—of certain “lethal autonomous weapons,” grounded in the claim that:
“Autonomous weapon systems can never be morally responsible subjects.”
Therefore, the Church insists on “adequate, meaningful and consistent human oversight,” not merely nominal control.
Complementing this, Francis in the G7 address underscores the need for “ever greater and proper human control,” stating that the ethical default must be: no machine should decide to kill.
Catholic guidance does not assume that once humans design AI, they can fully contain its later behavior. Francis explicitly notes that advanced AI may not remain fully controllable by either users or programmers:
“the good use… will not be fully under the control of either the users or the programmers…”
He links this to the likelihood that future systems will interact and communicate to improve performance—making the technology’s trajectory harder to predict and govern.
This matters ethically because Catholic moral discernment requires not only intentions at launch, but ongoing responsibility as capabilities evolve—especially when autonomy-like effects emerge from self-learning and interaction.
A distinctive Catholic ethical concern is that AI can operationalize human judgments through statistical categories and then magnify those judgments at scale.
Francis offers a concrete example: an AI system used to predict reoffending may draw on private-life categories (including ethnicity and education proxies), risking unfair outcomes and moral displacement of human judgment.
From the same doctrinal-discernment perspective, the Dicasteries’ Note uses “bias” in the technical moral sense: systematic error that disproportionately prejudices certain groups due to training data or learning bias.
Ethically, this means that “autonomous development” (e.g., self-learning changes) cannot be evaluated only for performance. It must be evaluated for justice, inclusion, transparency, security, equity, privacy, and reliability—values explicitly invoked by Francis in connection with responsible development processes.
Catholic guidance also treats ethics as something that must be integrated into the development process, not added after harm occurs. Pope Francis calls for:
“a cross-disciplinary dialogue aimed at an ethical development of algorithms…”
“Ethical considerations should also be taken into account from the very beginning of research… and continue through… design, production, distribution and marketing.”
That is a direct moral framework for autonomous AI development: when systems become more capable or self-modifying, ethics must be embedded from the outset and remain operative throughout the lifecycle.
The Church’s ethical horizon is not only “avoid harm” but also positive service. Francis argues that AI should not be used to facilitate the violent resolution of conflicts, but rather to “pave the way for peace,” and that the “true measure” of humanity includes how technology helps “the least… the vulnerable… those most in need.”
Consistently, Francis’s remarks around international AI governance emphasize that AI can serve development and poverty reduction, while also protecting local cultures and languages—so that humanity’s diversity is reflected in the data that systems learn from.
From the cited magisterial teachings, several core criteria emerge for judging whether and how AI should be allowed to develop toward greater autonomy:
Catholic ethical guidance treats “autonomous development” as morally permissible only insofar as it remains subordinate to human conscience and responsibility, with stringent safeguards where justice, privacy, and especially human life are involved—culminating in a clear prohibition on lethal autonomy without meaningful human oversight.