Too Much Control: from War to Health
Activity: Talk or presentation types › Lecture and oral contribution
Ezio Di Nucci - Other
Abstract: The relationship between control and technological innovation is attracting growing attention from both researchers and the media: from promises of more precision to fears of loss of control, the concept of control seems to be at the centre of debates around technology, whether it’s drones and autonomous weapons, self-driving cars, big data algorithms or medical AI. Most debates revolve around issues such as accountability, responsibility gaps, or exactly how much control we as humans need to keep – so-called meaningful human control, for example. Those are some of the questions I addressed in my most recent book, The Control Paradox: from AI to Populism (2020), where I argue that the very technological innovations meant to enhance our control might end up undermining it. But what if all the promises were fulfilled and we ended up with too much control? Apart from the self-control literature, the topic has been largely overlooked, so that one might ask: is too much control even possible to begin with? I introduce the concept of “too much control”, ultimately applying it to two diverse domains such as war and health, to explain why we ought to be careful what we wish for. In war, the basic problem with “too much control” is that it undermines chances of de-escalation: the worry being that it is our knowledge of the enemy’s strategic and technical limitations and deficiencies that leaves the door open for avoiding lethal counterstrikes. As the story of Stanislav Petrov reminds us, overestimating the enemy’s control over its weapon systems would have prevented de-escalation and resulted in a devastating nuclear counterstrike. In healthcare, the problem with “too much control” is that it increasingly allows for blaming health and disease on patients. Increase delegation of critical medical tasks to patients through intelligent technologies exposes vulnerable people to blame and guilt. Here the idea is that while these technologies might end up improving outcomes and lowering risk, when things don’t work out – and we all die in the end anyway – patients might blame themselves instead of nature or the healthcare system; and that might ultimately not be worth the gain in life-years.
1 Jun 2022