To reduce risk of AI getting out of control, machines should be made to doubt themselves, according to researchers at the University of California Berkeley.
In other words, make AI neurotic.
In a paper, researchers Dylan Hadfield-Menell, Anca Dragan, Pieter Abbeel and Stuart Russell say that one of the best ways to prevent AIs from misbehaving is to make sure we can turn them off. Even an AI that doesn't care about its own life will want to remain active -- or alive -- to make sure it can continue to be useful. So the solution is to make the AI doubt whether it's useful.
It is clear that one of the primary tools we can use to mitigate the potential risk from a misbehaving AI system is the ability to turn the system off. As the capabilities of AI systems improve, it is important to ensure that such systems do not adopt subgoals that prevent a human from switching them off. This is a challenge because many formulations of rational agents create strong incentives for self-preservation. This is not caused by a built-in instinct, but because a rational agent will maximize expected utility and cannot achieve whatever objective it has been given if it is dead. Our goal is to study the incentives an agent has to allow itself to be switched off.
It's not enough that the robots will take our jobs. They'll also be neurotic, constantly requiring reassurance.
My colleague Jamie Davis at Telecoms.com has some other ideas for personality types that can be programmed into AI to keep them in line:
Conceited Carl: As an AI application, Carl is no danger. Carl spends too long staring into the mirror to fix his hair, down the gym pumping iron or on ASOS trying to figure out what Jon from Love Island was wearing last night. This AI is so concerned about how it appears there is very little risk of world domination. Just tell it that its algorithm looks a little bit flabby and it will be on the virtual cross-trainer in no time.
Also, Hypochondriac Henrietta, Boring Benjamin, and more. Read it here: "How do you stop AI from taking over the world? Make it neurotic of course."
I fear I may be Boring Benjamin. Would you like to discuss the relative merits of Mac to-do list software?
— Mitch Wagner Editor, Enterprise Cloud News
Keep up with the latest enterprise cloud news and insights. Sign up for the weekly Enterprise Cloud News newsletter.