Author: Larry Lewis

A panel discussion on the international developments in artificial intelligence

Monday, April 1

1:30 pm-4:00 pm

CNA Headquarters

3003 Washington Blvd, Arlington, VA 22201

CNA and its Center for Autonomy and Artificial Intelligence invite you to join us for a discussion on international and adversarial developments in the area of artificial intelligence.

As the geopolitical paradigm shifts to an era of great power competition, the U.S. and peer competitors, including Russia and China, see AI and autonomous weapons as critical enablers that will provide a military edge in this new environment. With that in mind, the effective development of AI technology is vital for improving the competitive edge of the United States, the major goal of the 2018 National Defense Strategy.

To meet this goal, it is essential that U.S. national security professionals understand how our competitors plan to leverage AI and autonomous weapons. They must also remember that the U.S. is not in this fight alone. Through cooperation and interoperability with its allies, the U.S. can lead a combined research effort on AI. CNA analyst Samuel Bendett, an expert on Russia’s AI and autonomous weapons programs, will sit down with the Center for New American Security’s (CNAS) Elsa Kania, whose breakthrough research covers Chinese military innovation in emerging technologies including AI, and Jack Watling from RUSI, who will discuss initiatives within the UK. Together they will discuss how the U.S. can work with allies and outpace its peer-competitors in the field of AI. Join us at CNA for this discussion followed by a question and answer period.

For more information about this event or to RSVP please contact John Stimpson at stimpsonj@cna.org

This statement was made by CNA’s Director of its Center for Autonomy and AI to the UN’s Group of Government Experts (GGE) on Lethal Autonomous Weapon Systems (LAWS) in Geneva during its August 27-31 2018 meeting.

Ambassador Gill, thank you for your leadership in such a complex and critical issue, which is at the heart of the future of war. I have been part of these discussions since 2016, first as a diplomat and now as a scientist. Overall, we have seen some progress in the past few years. I point out the UK and Dutch positions which illuminate a broader framework for human control over the use of force.

Yet it is clear that there is a fundamental disagreement in this body on the way forward. Some have said today that this is because some States are stalling. But I offer another explanation.

Einstein said this: “If I had an hour to solve a problem, I would spend 55 minutes thinking about the problem and 5 minutes thinking about solutions.”

I believe this wisdom applies here. We have not had more progress in the past few years because we have not sufficiently defined the problem. States and other groups are still talking past one another.

One example. There are two types of AI: general and narrow. General AI is often described as an intelligent or superintelligent type of AI that solves many types of problems and does not, and may never, in fact exist. Narrow AI is a machine performing specific pre-programmed tasks for specific purposes. This is the type of AI we see in use today in many applications. Those two types of AI carry very different risks. When we fail to discriminate between the two types of AI in our discussions, we talk past one another and cause confusion. Framing this discussion around narrow AI – the kind of technology that is actually available to us now and in the near future – would help us to focus on the specific risks of AI and autonomy that need to be mitigated.

That is one example, and there are others. I discuss a number of such risks in CNA’s new report, AI and Autonomy in War: Understanding and Mitigating Risks .

Finally, there is much discussion of civilian casualties in this forum. It is surprising that, as a Group of Government Experts, we have not explored this particular problem more in depth. Such exploration is very possible. Having led many studies on how civilian casualties occur in military operations, there is much we as a group can learn about risks to civilians, how autonomy can introduce specific risks, and how technology can mitigate risk.

Overall, I believe we can learn from Einstein: this is a problem we can solve, but we still have work to do to adequately frame the problem.

This article is by CNA’s premiere expert in Russian unmanned and artificial intelligence developments, Samuel Bendett.

Russian state corporation “Russian Technologies (Rostec)”, with extensive ties to the nation’s military-industrial complex, has overseen the creation of a company with the ominous name – “Kryptonite” will work on creating “civilian IT products based on military developments in information security, including blockchain.” Specifically, Kryptonite’s work will involve “cryptography, machine learning, Big Data, quantum computing, blockchain and the security of telecommunications standards.”

This is interesting due to a fact that when it comes to such key information technology concepts, it’s the Russian state sector that is leading in research and development (R&D), not the civilian sector. In fact, as the announcement indicates, the military achievements will now be available to the Russian private sector, presumably aiming for eventual wide-spread domestic and international use. Russian Ministry of Defense (MOD) has made no secret of its desire to achieve technological breakthroughs in IT and especially artificial intelligence, marshalling extensive resources for a more organized and streamlined approach to information technology R&D. In fact, MOD is overseeing a significant public-private partnership effort, calling for its military and civilian sectors to work together on information technologies, while hosting high-profile events aiming to foster dialogue between its uniformed and civilian technologists.

“Kryptonite”s work is reminiscent of the eventual outflow of the American military information technologies to the civilian space during the Cold War, the Internet and cellular phone being most prominent examples. It also brings to mind the Soviet technological experience, when the military and security services had access to the nation’s best and brightest minds and talent, along with the first pick of major resources needed for advanced technology development. The eventual drawback was that such Soviet achievements were not eventually transferred to the civilian consumer sector that could have definitely benefited from the large-scale domestic adoption of such technologies. MOD’s current work on IT and advanced technologies is aiming to compensate for the lack of funding and development following the dissolution of the USSR and the tumultuous and cash-poor 1990s and early 2000s, when Russia imported numerous Western technologies for domestic and military use. Such policies created an uncomfortable dependence today that the MOD is currently aiming to overturn. While such efforts are at their beginning stages, there is reason to assume that today’s Russian military establishment is willing to experiment with a variety of technological approaches to develop domestic know-how and deliver it to the eventual consumer, military or civilian. One such approach is the creation of an “Era” “technopolis” that will work on various advanced technologies, including IT, blockchain and artificial intelligence – the effort is already underway and will be officially opened in September 2018.

“Kryptonite”’s importance as a military-civilian technological nexus was underscored by “Rostec” leadership, which stated that “…it is obvious that Russian military developments in information security have great potential for civilian use, including hi-tech exports. The creation of (this) joint venture is an excellent example of public-private partnership, and consolidation of efforts will increase synergy…while developing (IT) market competencies, as well as by incorporating the scientific and technical potential accumulated by Russia’s defense industry.” To achieve that, “Kryptonite” will work to “attract investments in order to commercialize innovative solutions and technologies created by the Russian defense industry’s enterprises and research institutes, as well as develop its own information security and Big Data competencies.”

As far as the company name – that probably wasn’t picked by accident. Russian government has long expressed concern that their reliance on imported IT products creates major security vulnerabilities. Developing domestic information technologies will help overcome that, while at the same time allowing Russian technology sector to eventually complete with American, Western and Asian hi-tech leaders. This technology race is only expected to accelerate – and Russian achievements merit close attention.

Samuel Bendett, an associate research analyst in the Center for Naval Analyses’ Russia Studies Program and a fellow in Russia studies at the American Foreign Policy Council, discusses the buzz surrounding Russia’s Hunter unmanned aerial vehicle, the failure of its Uran-9 unmanned ground vehicle program following secret combat testing in Syria, the outlook for Russian submarine and unmanned underwater vehicle innovation, and more during a June 27, 2018, interview with Defense & Aerospace Report Editor Vago Muradian in Washington. Check out the video here.

Last week, the UN again took up the issue of lethal autonomy in its Group of Government Experts on Lethal Autonomous Weapon Systems (or LAWS), a meeting of the Convention on Certain Conventional Weapons (CCW). Over the past few years, there has been a growing consensus within the CCW that human involvement in the targeting process is a solution to the risks posed by LAWS. With this consensus, the question then becomes: what is the precise nature of this human involvement? Often this involvement is described in terms of the decision to use the system, akin to a finger on the trigger. Thus human involvement becomes a human making the decision to pull the trigger.

An event at last week’s GGE argued that this view is too narrow. Entitled “The Human-Machine Relationship: Lessons From Military Doctrine and Operations,” the event was organized by CNA’s Center for Autonomy and AI and the University of Amsterdam. The event was attended by officials and diplomats, including the ambassador to the Netherlands. Dr. Lewis, CAAI Director, was joined by Merel Ekelhof, a Ph.D. researcher at the VU University of Amsterdam, and U.S. Air Force Lt. Col. Matt King.

Merel Ekelhof is known to the UN community as the person to first introduce the idea of human involvement spanning across the entire military targeting cycle, introducing the idea back in 2016. The event took that idea and showed specific ways which this human involvement can take. Lt Col King discussed legal requirements for the use of force and how this was satisfied in his personal involvement in targeting decisions in the Coalition Air Operations Center in Qatar in 2017. Then Dr. Lewis walked the group through several real world examples of incidents involving fratricide and civilian casualties, how humans are not infallible, and how broader human involvement outside of the trigger pull decision can create a safety net that can reduce the risks of autonomous systems.

As the meeting closed, Ambassador Gill (presiding over the GGE) suggested a model for human involvement that takes on this view of broader human involvement in the use of lethal force. The below picture (dubbed the “sunrise” chart by the UK delegation) shows his initial thoughts in this regard. CAAI is developing a follow-on report for the next GGE to be available at the end of August, where we will provide additional details and observations consistent with this broader view. Stay tuned!

For more information, see the CAAI report: Redefining Human Control.