Yahoo Finance Robots could take over as ‘algocracy’ threatens to replace democracy
Robots could use vast amounts of data and an insidious knowledge of ways to manipulate human behaviour to effectively take over vast swathes of our lives in what would effectively become rule by algorithm, or an "algocracy", the head of the City watchdog has warned.
Algorithms used by finance companies control which products customer are offered and how pricing works, but they may not even be fully understood by the executives using them, said the Financial Conduct Authority’s chairman, Charles Randell.
At the same time customers may not be aware of the data and control they are handing over when signing up to businesses’ terms and conditions.
Meanwhile firms are becoming more adept at using "nudges" to promote more custom, something which can be used for good – for instance, to prompt people to save more – but could also be used for malign purposes.
“The power of Big Data corporations and their central place in providing services that are now essential in our everyday lives raise significant questions about the adequacy of global frameworks for competition and regulation,” he said.
“The ordinary consumer may in practice have no choice in whether to deal with these corporations on terms which are non-negotiable and are often too general to be well understood. And without access to the data which consumers have signed – or clicked – away, new businesses may find it very difficult to compete.
“If you add all these factors together, they call into question the adequacy of the traditional liberal approach to the relationship between financial services firms and their customers. And regulation is central because it will help define whether artificial intelligence and Big Data liberate customers, or disenfranchise them.”
Creepy examples include reports in the US of credit card firms cutting borrowing limits when customers pay for marriage counselling, Mr Randell said, on the basis that divorcees are often a worse credit risk.
Meanwhile, some products, such as insurance, work by pooling groups of customers so that non-claimants’ payments cover the cost of payouts to claimants. But "big data" could enable companies to slice these groups down, only insuring the good risks and leaving some parts of the population with no cover.
On the other hand, there are more positive examples of people previously excluded from the financial system being now able to get insurance as sufficient data has been gathered to assess their risks more accurately.
Mr Randell recommends that companies remember they exist to serve people, and that they should seek to build trust by keeping customers properly informed to avoid a shock from bad behaviour being uncovered later.
