When Cardinal Robert Prevost took the name Leo XIV, the world understood that he was signalling his desire to become the heir of one of the most extraordinary Popes of the modern era.

Pope Leo XIII combined an astute metaphysical awareness with a determination to confront the raw power of political and economic development with the conscience of the Church, in order to defend those at the bottom of the political and economic heap who were being discarded by a particular economic revolution.

Leo XIV has now produced the encyclical Magnifica Humanitas, which immediately sets the present crisis before us in the form of competing anthropologies and competing eschatologies.

In the conviction that the human heart has been shaped by God to long for God more than for any other desire, the Pope has a duty to speak on God’s behalf to the hunger of the human heart, in order to awaken a deeper and truer perspective amidst the competing aspirations and temptations that every culture throws up.

There is little doubt that the Pope was right to offer his analysis of the dangers that the AI revolution presents. But we are still left with a pressing question: who is going to listen, and can any response now change a direction that has already been set in motion?

We might say that the Church has a role in speaking truth to power. The Church neither manufactures weapons nor writes software nor runs corporations. Instead, its task is to tell the truth about the human person and invite society to listen and to change. But who is going to listen, and how might we change?

Enough articles have already been written summarising the Pope’s concerns. The most interesting aspect of the encyclical’s presentation, however, seems to me to have been the presence of Christopher Olah, co-founder of Anthropic, who spoke at the Vatican presentation on May 25.

It was a stroke of political and pastoral genius to invite him. In the face of the hubristic ambitions of the AI industry, its extraordinary present achievements and the promise of almost unimaginable advances in the near future, Olah offered reasons from within the industry itself that mirrored the Pope’s critique from outside it. It is worth pausing to consider which reasons Olah chose to highlight.

First, given that AI threatens to displace human labour at scale, and so weaken the connection between human dignity and meaningful work, it raises profound questions about the value of human beings in society and about the relationship between work, identity and economic competence.

Second, it also raises the question of where our moral imagination concerning genuine human flourishing can still find authoritative guidance.

Third, Olah openly acknowledged the mysterious nature of AI models themselves – systems that even the researchers who build them often find deeply unsettling and do not fully understand.

Pope Leo had charted the territory, and Olah accepted the map, adding further details. He presented “rocks and dragons” beyond those that the Pope had drawn our attention to.

One of the most alarming parts of his speech was his realistic warning that every frontier AI laboratory – including his own – operates within a system of incentives and constraints that can conflict with doing what is right. Whether geopolitical, commercial or research-driven, these incentives push the industry relentlessly towards further development.

Perhaps the most troubling aspect of Olah’s contribution was his admission that many engineers inside these laboratories do not really understand what they have created. “I’m a scientist,” he said. “I lead a research team that studies the internal structure of these models – what is actually happening inside them. And I will be honest: it’s full of mysteries. We keep finding things that are mysterious, even unsettling.”

He went on to note that researchers are discovering structures within these systems that bear remarkable similarities to patterns identified in human neuroscience. “We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief and unease. I don’t know what it means, but I think it would be wrong not to pay attention.”

Acknowledging this mixture of ignorance and power, he added with notable humility: “If we want this technology to go well, it is enormously important that there be people outside those centres – people who care about things going well, who are paying attention, who are willing to say hard things, willing to be earnest, thoughtful critics.”

The philosophical background to the debate launched by Pope Leo involves two very different anthropologies and two very different eschatologies.

The Christian vision, rooted in Genesis, warns that there is a perpetual flaw in the balance between human aspiration and human competence. Humanity repeatedly stretches beyond its wisdom and overbalances itself. One of Christianity’s most important contributions to civilisation is the question: “Just because you can do something, are you prepared or even able to live with the consequences?”

On the other side of the moral fissure stands the determined optimism of the Enlightenment. Its softer version, inherited from Rousseau, expresses a perpetual confidence in human self-improvement. In contrast to its unfounded optimism that humanity is born as a blank slate, capable of infinite improvement, history has repeatedly exposed that confidence as dangerously naive. The 20th century ought to have buried such optimism beneath the evidence of its catastrophes, yet somehow it survives.

The harder version derives from Nietzsche, where power and will become dominant forces unconstrained by any shared moral order. We are therefore confronted by the possibility that the trajectory of AI research may become increasingly Nietzschean: driven by capability rather than wisdom, by achievement rather than restraint. Under the influence of ever more powerful forms of machine agency, technological growth, competence, speed and ambition may accelerate beyond our capacity to govern them, and remain unrestrained by ethics that are neither concrete nor enforceable. The result threatens to be the displacement, diminution and eventual marginalisation of the very human beings who invented the technology in the first place.

Pope Leo rightly insists that human beings are never means to an end and never disposable. Human flourishing depends upon the exercise of human agency in cooperation with God, constrained by the character and will of God, in a manner that safeguards the integrity of the gift of human nature itself.

We are left with two questions at this juncture:

Who is listening to Pope Leo?

Who is listening to Christopher Olah?

The constituencies driving artificial intelligence into the future may not easily be persuaded to restrain what can be done by appealing to what ought to be done. If the future of artificial intelligence becomes governed entirely by capability and no longer by moral responsibility, the damage done to human beings may prove unimaginably severe.

Yet Pope Leo ended the conference on a hopeful note: “What a great sign of hope it is that, despite our differences, we can listen to one another.” This interchange, he suggested, speaks to the possibility that wisdom and power need not become enemies.

The fact that the Pope and the head of Anthropic are willing to initiate a conversation in which the software industry is at least prepared to half-turn and pay attention to a voice offering both critique and restraint is undoubtedly a good sign.

But before we can answer the question of whether the Pope can either represent or exercise sufficient moral authority to influence a civilisation increasingly in the grip of technological power and possibility, we need first to identify who actually holds power in the context of software development.

The difficulty is that power is diffusely held. It is distributed across investors, corporations, governments, researchers, legislators and international institutions. Because it is so widely dispersed, it may lack the capacity to respond coherently to the questions Pope Leo has rightly articulated and placed in the public domain.

But what the Pope has achieved through his encyclical is to offer investors, software engineers, governments, legislative agencies and international regulatory bodies an opportunity to pause, to think, to reflect and, perhaps, to act. Catholics can take quiet pride in the fact that no other figure on the world stage possesses quite the same combination of resources, wisdom and moral authority required to open such a conversation.

The conversation has at least begun. Whether it deepens, and whether it reaches the people who most need to hear it, remains an open question.

It is, however, the right question to be asking.