Some Moral and Technical Consequences of Automation. Norbert Wiener (1960)
As machines learn they may develop unforeseen strategies at rates that baffle their programmers.
Man and Slave. The problem, and it is a moral problem, with which we are here faced is very close to one of the great problems of slavery. Let us grant that slavery is bad because it is cruel. It is, how- ever, self-contradictory, and for a reason which is quite different. We wish a slave to be intelligent, to be able to assist us in the carrying out of our tasks. However, we also wish him to be subservient. Complete subservience and complete intelligence do not go together…
When a machine constructed by us is capable of operating on its in- coming data at a pace which we can- not keep, we may not know, until too late, when to turn it off.”
Some Moral and Technical Consequences of Automation Author(s): Norbert Wiener Source: Science, New Series, Vol. 131, No. 3410 (May 6, 1960), pp. 1355-1358 Published by: American Association for the Advancement of Science Stable URL: https://www.jstor.org/stable/1705998 Accessed: 21-11-2019 23:13 UTC
Wikipedia. Norbert Wiener (November 26, 1894 – March 18, 1964) was an American mathematician and philosopher. He was a professor of mathematics at the Massachusetts Institute of Technology (MIT). A child prodigy, Wiener later became an early researcher in stochastic and mathematical noise processes, contributing work relevant to electronic engineering, electronic communication, and control systems. Wiener is considered the originator of cybernetics, the science of communication as it relates to living things and machines,[4] with implications for engineering, systems control, computer science, biology, neuroscience, philosophy, and the organization of society. His work heavily influenced computer pioneer John von Neumann, information theorist Claude Shannon, anthropologists Margaret Mead and Gregory Bateson, and others. Norbert Wiener is credited as being one of the first to theorize that all intelligent behavior was the result of feedback mechanisms, that could possibly be simulated by machines and was an important early step towards the development of modern artificial intelligence.[5]