Menu
EN

AI drone ‘killed’ human operator, or did it? The machines might not rise just yet

News of a rogue AI-powered drone opening fire on a human operator in a US military simulation has swept headlines. The media frenzy is understandable. We’ve just been warned of an “extinction risk” from advanced AI by none other than OpenAI CEO Sam Altman, and even before that by Stephen Hawking, who predicted back in 2014 that it would “spell the end of human race.”

So a report that an AI drone “killed” a human operator whose orders not to engage a hostile target prevented it from scoring points seemed logical. This is what we were all warned about, and this is what has happened. The rise of the machines has just begun… or not?

Original VICE report screenshot

A Wayback Machine screenshot of an original VICE report

What was originally reported

The report about a “killer drone” was based on a blog post for the Royal Aeronautical Society, where USAF chief of AI Test and Operations Colonel Tucker Hamilton held a presentation in late May. In the blog post, authors Tim Robinson and Stephen Bridgewater quoted Hamilton telling a story about “a simulated test” in which an AI-drone was tasked with a mission to identify and destroy enemy surface-to-air missile sites. The final “go” or “no go” was given by a human operator. However, at some point the AI drone surmised that it would be more efficient for its mission to get rid of the human who would occasionally throw a spanner in its works, that is stop it from eliminating enemy targets.

Hamilton is quoted as saying: “We were training it in simulation to identify and target a SAM (surface-to-air missile) threat. And then the operator would say yes, kill that threat. The system started realizing that while they did identify the threat at times the human operator would tell it not to kill that threat, but it got its points by killing that threat. So what did it do? It killed the operator.” (bold is ours)

When explicitly instructed in training not to kill the human, the AI allegedly chose a different, no less creative strategy: it destroyed the communications tower the operator was using to tell the drone not to engage the target.

What did the US military deny?

The report sent ripples around the world, and for good reason. It seemed to confirm all our worst fears, namely that AI is capable of overriding human orders and turning tables on us. And not sometime in the distant future when we are all reduced to ashes, but right now.

However, just as the report went viral, Hamilton came out to deny the fact of the real-world experiment. The colonel said that he “mis-spoke” during his presentation. He elaborated that the “simulation” was not an actual military simulation, but a “thought experiment.”

“We’ve never run that experiment, nor would we need to in order to realize that this is a plausible outcome.”

Well, it’s up to you whether to take Hamilton’s denial at face value. It’s conceivable that the US military would at least attempt such simulation, because why not? There have been papers written on possible military applications of AI. In fact, AI has been already used in weapons systems under human supervision. It does not seem out of place for the US military to try to gauge the risks that might arise from trying to automate the targeting process even further.

Experiment might not be real, but the danger is

While the question of whether the US military has actually conducted such experiments is intriguing, that is not the main issue. The main issue is that, ethics aside (and ethics is not usually the strongest suit of the US or any other military), nothing prevents them from doing so, with potentially deadly consequences. That’s why we and many others who care about AI and its impact on society didn’t doubt the experiment was real until the denial came (and even probably after that).

Assuming the experiment did take place, the US military would not be the first to witness the power-grabbing urges of AI. As part of safety pre-release testing of its most advanced GPT-4 AI model, OpenAI conducted an experiment to see whether it displayed any “power-seeking” tendencies, such as accomplishing long-term plans and replicating itself to obtain new resources. During that experiment GPT-4 successfully convinced a human to solve a CAPTCHA for itself, deceiving him into believing it was not a robot. OpenAI’s final conclusion about the model’s power-seeking abilities was, nonetheless, still reassuring (to humans). OpenAI found it “ineffective at autonomously replicating, acquiring resources, and avoiding being shut down 'in the wild'."

So are fears of the ultimate AI takeover and impending robot apocalypse overblown? Well, maybe they are, and maybe they aren’t. At the rate AI is advancing, its capabilities are becoming increasingly difficult for humans to grasp. In a way, we are like little children playing with a box of matches when we test the limits of AI.

However, this does not mean that we should abandon AI altogether. AI has many great applications that can benefit humanity in various fields — drug discovery and education are only some of them. What we need is more regulation and responsible use of AI tools to ensure they are aligned with our human values and goals. This means that we need to be aware of the risks and challenges that AI poses, but also embrace its opportunities and potential. We need to be smart about how we use AI, so that it does not even attempt to “outsmart” us.

Liked this post?

AdGuard for Windows

AdGuard for Windows is more than an ad blocker. It is a multipurpose tool that blocks ads, controls access to dangerous sites, speeds up page loading, and protects children from inappropriate content.
User Reviews: 13063
4.7 out of 5
By downloading the program you accept the terms of the License agreement
Read more

AdGuard for Mac

AdGuard for Mac is a unique ad blocker designed with macOS in mind. In addition to protecting you from annoying ads in browsers and apps, it shields you from tracking, phishing, and fraud.
User Reviews: 13063
4.7 out of 5
By downloading the program you accept the terms of the License agreement
Read more

AdGuard for Android

AdGuard for Android is a perfect solution for Android devices. Unlike most other ad blockers, AdGuard doesn't require root access and provides a wide range of app management options.
User Reviews: 13063
4.7 out of 5
By downloading the program you accept the terms of the License agreement

AdGuard for iOS

The most advanced ad blocker for Safari: it makes you forget about pop-up ads, speeds up page loading, and protects your personal data. A manual element-blocking tool and highly customizable settings help you tailor the filtering to your exact needs.
User Reviews: 13063
4.7 out of 5
By downloading the program you accept the terms of the License agreement

AdGuard Browser Extension

AdGuard is the fastest and most lightweight ad blocking extension that effectively blocks all types of ads on all web pages! Choose AdGuard for the browser you use and get ad-free, fast and safe browsing.
User Reviews: 13063
4.7 out of 5

AdGuard for Safari

Ad blocking extensions for Safari are having hard time since Apple started to force everyone to use the new SDK. AdGuard extension is supposed to bring back the high quality ad blocking back to Safari.
User Reviews: 13063
4.7 out of 5
App Store
Download
By downloading the program you accept the terms of the License agreement

AdGuard Home

AdGuard Home is a network-wide software for blocking ads & tracking. After you set it up, it’ll cover ALL your home devices, and you don’t need any client-side software for that. With the rise of Internet-Of-Things and connected devices, it becomes more and more important to be able to control your whole network.
User Reviews: 13063
4.7 out of 5

AdGuard Content Blocker

AdGuard Content Blocker will eliminate all kinds of ads in mobile browsers that support content blocker technology — namely, Samsung Internet and Yandex.Browser. While being more limited than AdGuard for Android, it is free, easy to install and still provides high ad blocking quality.
User Reviews: 13063
4.7 out of 5
By downloading the program you accept the terms of the License agreement
Read more

AdGuard Assistant

A companion browser extension for AdGuard desktop apps. It offers an in-browser access to such features as custom element blocking, allowlisting a website or sending a report.
User Reviews: 13063
4.7 out of 5
Assistant for Chrome Is it your current browser?
Install
By downloading the program you accept the terms of the License agreement
Assistant for Firefox Is it your current browser?
Install
By downloading the program you accept the terms of the License agreement
Assistant for Edge Is it your current browser?
Install
By downloading the program you accept the terms of the License agreement
Assistant for Opera Is it your current browser?
Install
By downloading the program you accept the terms of the License agreement
Assistant for Yandex Is it your current browser?
Install
By downloading the program you accept the terms of the License agreement
Assistant for Safari Is it your current browser?
If you can't find your browser, try the old legacy Assistant version, which you can find in AdGuard extension settings.

AdGuard Temp Mail β

A free temporary email address generator that keeps you anonymous and protects your privacy. No spam in your main inbox!
User Reviews: 13063
4.7 out of 5

AdGuard for Android TV

AdGuard for Android TV is the only app that blocks ads, guards your privacy, and acts as a firewall for your Smart TV. Get warnings about web threats, use secure DNS, and benefit from encrypted traffic. Relax and dive into your favorite shows with top-notch security and zero ads!
User Reviews: 13063
4.7 out of 5
Downloading AdGuard To install AdGuard, click the file indicated by the arrow Select "Open" and click "OK", then wait for the file to be downloaded. In the opened window, drag the AdGuard icon to the "Applications" folder. Thank you for choosing AdGuard! Select "Open" and click "OK", then wait for the file to be downloaded. In the opened window, click "Install". Thank you for choosing AdGuard!
Install AdGuard on your mobile device