Issues on combining human and non-human intelligence

Abstract

The purpose here is to call attention to some of the issues confronting the designer of a system that combines human and non-human intelligence. We do not know how to design a non-human intelligence in such a way that it will fit naturally into a human organization. The author's concern is that, without adequate understanding and consideration of the behavioral and psychological limitations and requirements of the human member(s) of the system, the introduction of artificial intelligence (AI) subsystems can exacerbate operational problems. We have seen that, when these technologies are not properly applied, an overall degradation of performance at the system level can occur. Only by understanding how human and automated systems work together can we be sure that the problems introduced by automation are not more serious than the problems solved.