That takes up a big chunk of the worry space: that there is one artificial agent that becomes very powerful, and is therefore able to shape the future according to its goals. Those goals may not have anything to do with human goals. They may be arbitrary.
This statement got me thinking. Everyone is worried that A.I. will take over and make us do things we don’t want to do. But don’t we already have big, very powerful agents shaping the future according to their own goals? Government agencies come to mind. Dictators. Corporate management. Even parents could be categorized this way . And yet these agents are not inherently all-powerful. They can’t physically make us do anything. Still, we do what they want. Because society tells us we must. In theory, isn’t artificial intelligence just a different aspect of this same societal issue we deal with everyday?
Read more from New Republic via Pocket