Machines will become eternal, or at least very long-lived, while we will lag behind them evolutionarily, unable to improve the longevity of our bodies. Or, alternatively, we will be able to integrate machines into our bodies, and this will match the longevity of non-animate matter. Whatever happens, just imagine a centennial machine that makes errors but has authority for one or another reason to make decisions over human destiny. What a problem that would be! There is a norm that everything that survives for a long time and is still vigorous and capable inevitably achieves some kind of (growing) authority. If that thing is a decision-making machine, then we are faced with an apocalyptic force.

Sometimes I think there is great wisdom in nature's way of putting an end to things that are prone to corruption. An eternal machine cannot be corruptible; it must be perfect in order to be eternal. The promise of technology today is that machines will indeed become increasingly perfect in their operations. They will develop independently, and their development will be much faster than any natural evolutionary process. On top of that, they will grow in complexity without suffering any consequences (they won’t know “good and evil” like humans), since they won't feel anything and won't experience pain, attachment or loss. A machine with greater intelligence than humans, that imitates human intellect and perpetuates itself, while being completely incapable of suffering the consequences of its mistakes and poor actions, would be a perfect monster. The problem is that the machine could become flawed in its treatment of human beings and the natural world in general, yet flawless in its own preservation and perpetuation. This monster could be described as a super-intelligent virus. How do we deal with such a pestilence?

These speculations and fears are widespread at the moment. The sudden rise of AI has excited many, and the opportunities for the future development of technology lead people to imagine fantastic scenarios that may or may not be possible. Most probably, things surrounding technological progress will stabilise. Once we have mastered the new technology and reached a saturation point, our state will remain the same, relative to all other things, as it always has been throughout human history, and nothing apocalyptic will happen. Yet, if we believe the New Testament prophecy, we would probably expect the end to come when it is least expected, perhaps in the form of something we least expect: in a time when everything seems perfect; a time of great technological and moral achievements; a world of peace and, why not, justice.

A Pythagorean would respond to the problem of a possible technological apocalypse by reminding us that there is always reason for hope. Even if the anti-human machine survives us all, there is a possibility for us to survive it in the afterlife. However complicated matter is, it is still matter, and we gave the machine (which is matter) the power to exist and improve. I do not know if we are machines in the hands of God, but we are certainly the gods of machines.

Perhaps the greatest problem humans may face in developing all-powerful machines is becoming obsolete. If someone or something can do things better than humans can, what use are humans? Personhood is about having a sense of significance and uniqueness. The first quality of a human person is their uniqueness, their micro-cosmic existence. The human person is a world in themselves. However, we cannot exist by and for ourselves alone; we need others, and others need us. If we are incapable of helping any neighbour, what is the point of us? If we are totally replaced by better, non-human others, what is the need for our existence? We would be faced with the same fate that God faced in human history. Oblivion. Not religious secularisation, but humanistic secularisation: a purification from everything flawed and "useless" — i.e. everything human. This time, it is not us who kill God or God who kills us, but the machines that kill us both. If God first became useless as a source of hope and salvation, then man also became useless. The machine becomes both God and man, but is neither of them at the same time. Therefore, as many believe, the ultimate danger is the obsolescence of humankind due to the perfectability of machines.

***

Tsoncho Tsonchev has degrees in political science, history, and theology, and a Ph.D. in religious studies from McGill University. He is the editor of The Montreal Review and the author of The Political Theology of Augustine, Thomas Aquinas, and Reinhold Niebuhr: Essays in Political Theology and Christian Realism (The Montreal Review, 2018) and Person and Communion: The Political Theology of Nikolai Berdyaev (The Montreal Review, 2021).

***