Yahoo Finance ‘Racist’ self-driving cars may not spot dark-skinned faces, Law Commission warns
‘Racist’ self-driving cars may not spot dark-skinned faces when pedestrians cross the road, the Law Commission has warned.
The Law Commission is drawing up the legal framework for self-driving cars in anticipation of their use being ramped up over the next decade.
But the commission has warned that self-driving cars “may struggle to recognise dark-skinned faces in the dark”.
Previously, bias “has crept into the design of vehicles and automated systems”, the commission said in its joint consultation with the Scottish Law Commission.
“Air bags save many lives, but the first generation of air bags posed risks to smaller passengers, such as women of small stature, the elderly, and children, because they were developed with adult males in mind.
There is a risk that the technology could also discriminate against women.
“Current facial recognition software may also exhibit a bias towards white, male faces. For non-white and non-male faces, the accuracy of facial recognition systems may decline significantly,” the consultation said.
The Department of Transport has said that by 2035 more than a third of new UK cars sold could have self-drive capabilities. The government has invested over £200m into the cars’ research and development and predicts more than 40,000 jobs could be created because of the growth in the industry.
The industry is predicted to have a total market value of £41.7bn by 2035.
A 2019 study from the Georgia Institute of Technology found that self-driving cars were more likely to drive into black people because the sensors and cameras used in the cars are better at detecting people with lighter skin tones. The cars were less likely to spot black people and to stop before crashing into them, the authors found.
The Law Commission has consulted on creating legal requirements for the cars and it is due to publish its final report towards the end of 2021.
In its consultation the Commission warned that in the future there are several ways self-driving cars could negatively impact the lives of some people.
“Systems may not have been trained to deal with the full variety of wheelchairs and mobility scooters; or they may struggle to recognise dark-skinned faces in the dark.
“If systems are designed to recognise pedestrians through leg movements, those movements may not be as pronounced for people wearing long skirts or robes.
“Where designers are drawn predominantly from one demographic group (such as young men) it is easy for the diversity of those affected by the design to be overlooked.”
In response to the consultation several developers argued that it would be impossible to ensure that self-driving cars always treated everyone equally.
But the Commission said it is important to show that self-driving cars have been trained using a broad range of people who are representative of the population in the areas where the cars will be driving.
Self-driving cars bring ethical questions “to the fore”, the commission said, and regulators will have to consider how risk is distributed between one group and another.
“This includes questions of how to justify the imposition of harm on some more than others. In future it may be necessary to consider an explicit requirement to address the ethical dimension of such changes.”
Speaking to the Sunday Times Edmund King, AA president, said: “Technology is not a prison. We must embrace technology as it can save lives. However, when it comes to autonomous cars, that technology must be accurate, precise and non-discriminatory. Human error is a factor in a majority of crashes but we shouldn’t just transfer the risks and accept robot error.
“The last thing we need is the next generation of Mondeo Man being a racist, misogynist self-driving automobile. These technological hurdles need to be overcome before drivers can take their hands off the wheel.”
