Australasian Science: Australia's authority on science since 1938

Who to kill? The dilemma of driverless cars


Driverless cars hold the promise of safer transport. But how should they react when loss of life appears inevitable? Should a car swerve to miss a pedestrian on the road, even if doing so would kill the passenger?

New US research, published in Science (, explores this ethical dilemma in a series of surveys, revealing that people generally want automated cars to be utilitarian (i.e. prevent the greatest loss of life) but, when pressed, admit that they would prefer to buy a driverless car that protects the driver at all costs.

The authors say regulation may provide a solution to this problem, but that their results suggest this could cause delays in the adoption of AVs and further deaths in the long run. "The lives saved by making AVs utilitarian may be outnumbered by the deaths caused by delaying the adoption of AVs altogether," said the authors.

"The study sheds some light on the state of public sentiment on this ethical issue. It shows that aligning moral AI driving algorithms with human values is a major challenge – there is no easy answer!

What I found interesting in this research is that participants were reluctant to accept government regulations of utilitarian AVs – in fact, the surveys showed that participants would be less likely to consider purchasing an AV with such regulation than without.

This to me is even a bigger challenge: (1) Deciding whether governments should regulate algorithms, (2) What tests and procedures should be put in place to ensure that the algorithms are compliant? After all, these are very rare events and such instances are not routine. Therefore, lacking a large set of examples, they are relatively resistant to training or programming

We also need to recognise that mobility and travel (whether by car, train, bus, aircraft etc.) are inherently risky, and can never be safe. We take calculated risks every time we travel.

With AVs, however, there seems to be some hyped expectations that they should be perfectly safe.
They won’t be and I believe the situation is not as complicated.

For example, plenty of ethical decisions are already being made in automotive engineering today. Inherent in airbags, for example, is the assumption that they are going to save a substantial number of lives, and only kill a few.

Some people have even gone further to suggest that given the number of fatal traffic accidents involving human error today, it could be considered unethical to introduce self-driving technology too slowly.

The biggest ethical question then becomes: How quickly should we move towards full automation given that we have a technology that potentially could save a lot of people, but is going to be imperfect and is going to kill a few? Should they be allowed on our roads, even if they make such mistakes?

It is difficult to speculate what this would mean for Australia, because the results are based on surveys of U.S. residents. I suspect that the public sentiment would be similar though. We do need to engage with the public on this.

I feel there is a leadership vacuum in this public policy space in Australia, and we need better engagement with the community to clarify the issues, concerns and expectations and lead in informing and shaping the future policies in this space."

Hussein Dia is an Associate Professor in Transport Engineering at the Swinburne University of Technology

"Studies like this are to be welcomed. They illustrate the challenging ethical dilemmas we face as AI systems like autonomous cars get fielded.

Given that driverless cars are less than a decade away, we need to work out, as a society, how we program such systems. Unlike the past, where if you survived an accident, you could be brought in front of the courts if you drove irresponsibly, we will have to program computers with behaviours in advance that determine how they react in such situations.

I would, however, caution the results that can be taken away from studies like these undertaken on Amazon Turk where participants are not themselves under any danger and had plenty of time to decide what the system should do. This may not reflect how we would, as drivers of cars, act in such moments of crisis.

Nevertheless, it is good to see such work, for the uptake of driverless cars will have a profound benefit on society, reducing road deaths greatly, and liberating many groups like the elderly and the disabled who are currently denied personal mobility."

Professor Toby Walsh is the Research Leader of the Optimisation Research Group at CSIRO's Data 61 and the University of New South Wales, Australia

"The issue raised by this paper is an important one but perhaps not the only one. We need to decide how much control we give to machines and as machines are used more extensively in our lives this question becomes more central.

"This is an example of a global issue which is best studied using a transdisciplinary approach. More work needs to be done by social scientists, computer scientists, engineers, insurance companies and those involved in legislation.

"The challenge raised in the paper is similar to the decision a military jet pilot would need to make when the plane is crashing and there is a risk of the plane crashing in a residential area and whether or not the pilot(s) risk their lives to continue to fly the plane out of the residential zone. The possibility of cases such as this happening in aviation is not high. The same may be true in the case of driverless cars.

"Many automotive companies and universities have already started research aimed at studying this life and death challenge. Although driverless cars will need to substantially reduce the risk of accident situations such as what is raised in this paper, such issues would need to be resolved before driverless cars become common on our roads. When we use artificial intelligence we are trusting a machine to make decisions for us. Trading shares, driving cars and flying airplanes are examples of such cases."

Prof Hossein Sarrafzadeh, Director, Centre of Computational Intelligence for Cyber Security and High Tech Transdisciplinary Research Network, Unitec Institute of Technology

"There is a built in trigger, that we fear the unknown and don’t trust science whatever the expert opinion and scientific studies. This is one of the reasons when the Docklands Light Railway system was introduced in 1987, autonomous safety fears meant each train had a safety operator. Driveless trains now operate in many cities and can be seen in most international airports connecting us between terminals. We already have autonomous pizza delivery systems.

"Autonomous vehicles will reach a tipping point where the advancement in science, the economic arguments and technology get to a point of incremental change and human consciousness ‘that it is going to happen’ and ‘will happen’. First of all we will see a series of small steps. Watch out for uber autonomous taxis in Pittsburgh or autonomous ships or autonomous cargo planes.

"We always fear the future, but without science and advancement we would still be in the cave and the wheel not have been invented."

Assoc Prof Ian Yeoman, School of Management, University of Victoria Wellington