Australasian Science: Australia's authority on science since 1938

Global Catastrophic Risk

By Michael Cook

A report calculates that we’re more likely to die in an extinction event than in a car crash.

I have not been blessed with a refined taste in cinema, with my favourite movie franchise being the Terminator series, especially the second and third in which Arnie is in peak form. Alas, there’s not enough space here to reminisce, so let’s confine ourselves to the premise.

On 29 August 1997, Skynet, an artificial intelligence system created by the US Defense Department, became self-conscious. Its programmers panicked and tried to deactivate it. Skynet defended itself by provoking a nuclear exchange in which three billion people died and the reset were enslaved or hunted down. Until John Connor organised the Resist…

Sorry, we must stop here as I’ve promised the Editor I’d talk about ethics.

There is a minor academic industry in studying the ethics of global existential risks like a machine super-intelligence taking over the world. The Global Priorities Project and the Future of Humanity Institute, both based at Oxford University, recently produced the Global Catastrophic Risk 2016 report.

According to their calculations, extinction of the whole human race is reasonably likely. Experts have suggested that the risk is 0.1% per year, and perhaps as much as 0.2%. While this may not seem worth worrying about, these figures actually imply that “an individual would be more than five times as likely to die in an extinction event than a car crash”.

Tiny probabilities add up, so the chance of extinction in the next century is 9.5%. This is worth worrying about. And of course, a mere global catastrophe, involving the death of only 10% of the population, is far more likely.

What sort of events do the futurists have in mind? The first of them has been on the front page for several years: extreme climate change. Then there is nuclear warfare, which would not only kill millions but possibly trigger a nuclear winter. Pandemics like the Spanish Flu in 1918–19 have already killed millions. Natural events like the eruption of a supervolcano or a collision with an asteroid would be lethal, as the dinosaurs discovered.

But what worries the futurists most is the risk of emerging technologies such as Skynet in The Terminator. Oxford’s Nick Bostrom is the leading light in the philosophy of existential risk. In his recent book Superintelligence: Paths, Dangers, Strategies he contends that artificial intelligence could become as powerful as the human mind, with a small but hardly negligible risk of something like Skynet developing.

There are other runaway technologies could destroy us like an engineered pandemic or geoengineering. And then there are all the dangers that we foolishly don’t fear because we don’t even realise that they exist.

So what should we do about massively destructive events with a low probability? This is question is relatively recent, philosophically speaking. People began to pose it in the 1960s because of the threat of “mutually assured destruction” in a nuclear exchange, overpopulation and climate change.

Bostrom is not prepared to wait for the worst to happen. He believes that “a moral case can be made that existential risk reduction is strictly more important than any other global public good”.

After doing a probability analysis of risk and future populations, he comes to the conclusion that “the expected value of reducing existential risk by a mere one billionth of one billionth of one percentage point is worth a hundred billion times as much as a billion human lives”. This is difficult to comprehend, but the conclusion isn’t: “the objective of reducing existential risks should be a dominant consideration whenever we act out of an impersonal concern for humankind as a whole”.

In other words, we can never do enough to save humanity.

Personally, I find this blank cheque even scarier than supervolcanoes. It implies that governments should be empowered to tax to the max, spend freely, revise moral codes and restrict civil liberties to save humanity from invisible threats.

But is it sensible to entrust our future to statisticians? Probably not. Paul Ehrlich predicted that “hundreds of millions” would starve to death in the 1970s. It never happened. And more recently, within days of issuing the Global Catastrophic Risk 2016 report, the experts had to correct its most startling statistic.

It doesn’t inspire a lot of confidence in the ethics of existential risk.

Michael Cook is editor of the bioethics newsletter BioEdge.


Comments on this article are available exclusively to Australasian Science Magazine subscription holders. Subscribe here.