Redefining Human Control: Event at the UN’s Meeting on Lethal Autonomy

Last week, the UN again took up the issue of lethal autonomy in its Group of Government Experts on Lethal Autonomous Weapon Systems (or LAWS), a meeting of the Convention on Certain Conventional Weapons (CCW). Over the past few years, there has been a growing consensus within the CCW that human involvement in the targeting process is a solution to the risks posed by LAWS. With this consensus, the question then becomes: what is the precise nature of this human involvement? Often this involvement is described in terms of the decision to use the system, akin to a finger on the trigger. Thus human involvement becomes a human making the decision to pull the trigger.

An event at last week’s GGE argued that this view is too narrow. Entitled “The Human-Machine Relationship: Lessons From Military Doctrine and Operations,” the event was organized by CNA’s Center for Autonomy and AI and the University of Amsterdam. The event was attended by officials and diplomats, including the ambassador to the Netherlands. Dr. Lewis, CAAI Director, was joined by Merel Ekelhof, a Ph.D. researcher at the VU University of Amsterdam, and U.S. Air Force Lt. Col. Matt King.

Merel Ekelhof is known to the UN community as the person to first introduce the idea of human involvement spanning across the entire military targeting cycle, introducing the idea back in 2016. The event took that idea and showed specific ways which this human involvement can take. Lt Col King discussed legal requirements for the use of force and how this was satisfied in his personal involvement in targeting decisions in the Coalition Air Operations Center in Qatar in 2017. Then Dr. Lewis walked the group through several real world examples of incidents involving fratricide and civilian casualties, how humans are not infallible, and how broader human involvement outside of the trigger pull decision can create a safety net that can reduce the risks of autonomous systems.

As the meeting closed, Ambassador Gill (presiding over the GGE) suggested a model for human involvement that takes on this view of broader human involvement in the use of lethal force. The below picture (dubbed the “sunrise” chart by the UK delegation) shows his initial thoughts in this regard. CAAI is developing a follow-on report for the next GGE to be available at the end of August, where we will provide additional details and observations consistent with this broader view. Stay tuned!

GGE pic on human control

For more information, see the CAAI report: Redefining Human Control.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: