How Should Humans Collaborate with Machines? Experimental Evidence for Image Classification Tasks

We report results from a series of experiments where human subjects and a state of the art artificial intelligence (AI) should classify and label images. The AI consistently outperformed the human subjects. When humans used the AI for decision support, their performance increased substantially. However, they were unable to beat the AI.

In one experimental condition called “inversion”, we turned the hierarchy on its head. Now, the AI would routinely classify images and it would delegate tasks to the human subjects if it did not know the answer. This lead to the overall best results. Inversion was the only experimental condition where a team consisting of humans and AI could outperform AI working alone.

We explain the results by showing that humans and AI had complementary skills. In order to profit from a collaboration, each side had to understand their skills owned and missing. The AI was much better at this. It could profit even from the worst-performing humans, but humans had difficulties delegating the “right tasks” to the machine.

The results raise important questions about the future of work: If inversion outperforms classic decision support on a broad scale, do we want it as a society? If the answer is yes, how do we grant AI adjustable amounts of autonomy? If the answer is no, who will tap into the unrealized potential, and what will happen to businesses that do not want to invert?