Anthropic and Claude warn of the “recursive self-improvement” of artificial intelligence and its serious implications for humanity
Anthropic, the company behind Claude, issued a warning on June 4 about the rapid, recursive self‑improvement of AI and the serious risks it poses. The warning follows Pope Leo XIV’s Magnifica Humanitas, illustrating a convergence of concerns between the Vatican and Silicon Valley. Anthropic’s message stresses that AI may advance faster than society can govern it, raising urgent questions about governance and safety. The article notes that such cautions were previously mainly voiced by philosophers, ethicists and religious leaders, but are now also coming from industry insiders.
4 months ago
Artificial intelligence’s rapid self‑improvement is prompting warnings from both tech insiders and the Vatican, highlighting shared concerns about governance, moral guidance, and the preservation of uniquely human capacities. 1
Anthropic, the creator of the Claude model, issued a public warning that AI systems may soon be able to design increasingly powerful successors with minimal human input, a process known as “recursive self‑improvement.” 1
The company notes that its own models already perform a growing share of the coding work needed for new AI generations, and that the pace of autonomous capability is accelerating dramatically. 1
Anthropic stresses that fully autonomous self‑improvement has not yet occurred and may never happen, but it urges governments and firms to consider mechanisms for slowing or pausing the development of the most advanced systems to avoid a governance gap. 1
Just weeks after Pope Leo XIV released Magnifica Humanitas, his first major document on AI, he warned against building a new “Tower of Babel”—a powerful technological culture detached from a shared moral vision. 1
Leo XIV argues that the central question is not whether innovation should continue, but whether humanity can ensure that AI serves authentic human flourishing and respects moral responsibility, wisdom, and dignity. 1
Moral theologian Charles Camosy, who collaborates with Anthropic, interprets the Pope’s call to “disarm” AI as an appeal to prevent technology from dominating the human person rather than a rejection of progress. 1
A new study by the Consortium for Faith and Ethics in AI, involving researchers from Baylor, Brigham Young, Notre Dame, and Yeshiva Universities, found that leading language models—including those from Anthropic, OpenAI, Google, Meta, and xAI—largely omit religious perspectives when answering ethical questions. 1
The “All Faith Benchmark” revealed measurable differences in how various religious traditions are represented, raising concerns about bias, fairness, and cultural understanding in AI outputs. 1
Both Anthropic and the Vatican acknowledge that effective regulation would require unprecedented international cooperation; otherwise, any slowdown could be undermined by nations or corporations seeking a strategic advantage. 1
The convergence of warnings from a leading AI firm and the Pope underscores a growing consensus that AI development must be paired with deep reflection on anthropology, ethics, and the meaning of human dignity. 1*
Examine Catholic teachings on technological stewardship and human dignity
Catholic teaching treats technology neither as a neutral instrument nor as an autonomous force. Technology must be stewarded so that it serves human dignity, the common good, and responsible care for creation, because technological power always takes on moral meaning through the purposes and choices of persons and societies.
Catholic doctrine does not endorse either a simplistic technophilia (“technology automatically saves”) or technophobia (“technology is inherently bad”). Instead, the Church affirms both truths:
This is why the Church frames the question not as “yes or no to technology,” but as a choice of what vision of the human person and society technology will embody. Pope Leo XIV summarizes it in the language of “Babel” versus “Jerusalem,” where the central issue is whether technology is aligned with fraternal coexistence and God.
A central Catholic criterion is explicit: intrinsic human dignity must be the “key criterion” for judging new technologies, and technologies are ethically sound to the extent that they respect that dignity and help it be expressed more fully.
The Church’s logic is that technology directly affects persons and social relations; therefore it cannot be assessed only by efficiency or innovation. Pope Francis emphasizes that technological progress needs “an adequate development of responsibility and of values” so its benefits can truly serve people rather than threaten their dignity (for example, through injustice or social exclusion).
The Catholic moral tradition also insists on the finality of technology: it exists for the human person, and cannot be used as if persons were interchangeable inputs. The Church’s doctrinal reasoning applies to the whole domain of morally sensitive technologies: technology “must respect [man’s] finality” and is never free to pursue every possible intervention independent of morality.
Pope John Paul II’s framing is anthropological and warns against a reduction of the human person to what empirical methods can measure—especially when technology is applied “prescinding from the basic anthropological and ethical parameters.”
Catholic social teaching identifies a deeper temptation: the technocratic paradigm, where the logic of efficiency/control/profit becomes the standard by which decisions are made. When that logic dominates, technology stops being merely a tool and begins to dictate what “can be discarded,” reducing creation to exploitation and people to “mere cogs.”
Pope Leo XIV likewise warns that if the human being is treated as something to be “perfected or surpassed,” it becomes easier to accept “necessary sacrifices,” placing burdens on the most vulnerable.
Catholic teaching places a major responsibility on public authorities and lawmakers. Pope Francis teaches that policies and regulations are not an anti-technology stance; they exist to protect human dignity when threats emerge from innovations operating without social responsibility.
He cites concrete risks that demand legislative attention, including:
The aim is that regulation should concentrate on promoting integral human development and peace, rather than “progress as an end in itself.”
Pope Francis stresses that a responsible ethical framework must grow alongside technical capacity—otherwise technology becomes dangerous.
In 2020, he even proposed a “new discipline” of “algor-ethics”: ethical development of algorithms so there can be competent review of how technologies shape the relationships between human beings and society. Importantly, he notes that the Church’s social teaching—dignity, justice, subsidiarity, and solidarity—should guide this ethical development.
When Pope Francis gives the example of robots, he describes a dual risk:
So Catholic stewardship evaluates the social effects of technology—not only productivity.
Catholic teaching treats human dignity in cyberspace as requiring respect for privacy and protection from intrusive surveillance and unauthorized access to personal information.
The Church’s 2024 teaching (Dignitas Infinita) directly links the stewardship of technology with proactive defense of human dignity, mentioning the need to address digital trends that harm persons and peace, and recalling that media should guide toward truth, service, closeness to the underprivileged, and the common good.
Pope Francis also highlights that technology creates “order in social relations” by arranging power: it can enable some people to act while preventing others from acting. Therefore, the moral evaluation cannot ignore how technology redistributes agency and influence.
A recurring theme across the sources is that technological progress requires parallel moral-social progress. Pope Francis states that benefits depend on whether new possibilities are used ethically, and that this requires developing responsibility and values alongside technical progress.
Pope Leo XIV expresses the same principle by warning that the decisive issue is not technology “as such,” but the underlying vision—particularly the moral-social progress needed so that advances do not turn against humanity.
Catholic technological stewardship is ultimately anthropological: it treats every technological decision as a moral decision because technology both expresses and shapes how humans understand themselves and one another. Human dignity must be the decisive criterion for evaluating technology; governance must protect dignity and the common good; and new digital systems must be developed with justice, privacy, truth, inclusion, and moral responsibility in view.