Feb 28 2008
I for one welcome our new robotic overlords.
University of Sheffield professor and computer scientist Noel Sharkey, best known for his appearances on the BBC show Robot Wars, in a talk before Britain’s Royal United Services Institute warned the world that automated military robots “pose a threat to humanity.” I agree. Seriously.
Well, OK – not right now. But it is not too early to think about the implications of developing increasingly automated robots designed for warfare. While I think it is an unlikely scenario that such machines will take over the world anytime this century, as in The Matrix or Terminator, they may pose a credible risk in the near future.
Sharkey warned that such machines may fall into the hands of our enemies. They could then be hacked, reprogrammed, or just taken control of and used against our side. Or they can be copied. We may see cheap Chinese knockoffs of our latest warbot showing up on the battlefield. This may in turn lead to an arms race of robotic warfare – and that can only lead to Skynet blowing up the world.
Even if it does not get that far, we still may have a world threatened by hundreds of millions of killer robots, some of which could be in the hands of terrorists or psychotic dictators. Of course you could say this about any weapon. Nuclear weapons are inherently dangerous because they may fall into the hands of dictators or terrorists. But this just reinforces the point, as the former has already happened and there are serious concerns about the potential of the latter.
Another cause for concern is malfunction. A killer robot, armed and armored, and fully automated with subroutines that basically involve killing human beings and destroying infrastructure, might fall prey to what is technically called a “glitch.” Most computer users are familiar with this phenomenon. The difference between engaging the enemy and going on a bloody rampage may be a fine line.
Any such system would, of course, have fail safes – manual override, a kill switch, redundancies, and a self-destruct if all else fails. But fail safes only reduce the risk of error, they can never eliminate it (or reduce the risk to zero). And war is often chaotic and unpredictable. The best made-plans of our military leaders may break down on the battle field, in the hands of a panicky private, or in the midst of a hasty retreat. No one can see all ends.
What are some options to prevent what I will call the Cylon scenario? Well, we could never build such things. That would require an international treaty limiting the development and deployment of automated robots designed to kill or destroy. (The legal and technical mavens can work out all the details as to how to define this.)
Or – we could develop robots for warfare that are not automated but that always require a human driver (even if they are remote). They are simply incapable of “pulling the trigger” on their own. There is gray here as well, though. Can one human driver coordinate dozens of robots – like playing a video game?
I predict that we and others will develop some type of automatic robots for various aspects of warfare. They already have, in fact, and I see no reason why it will slow down. They will use the “fail safe” argument for defense, and I can only hope that there is sufficient oversight to truly minimize the probability of disaster. I will also note that for now I am much more worried about rogue nukes than killer robots.
But there is one line I recommend we never cross – developing truly artificially intelligent, automated, mobile military robots. Especially ones that are independent, meaning they can repair themselves and secure their own power source. And especially especially ones that can replicate themselves or build new robots. One could argue that any independent robotic AI is a long term risk. Why should we seed the universe with entities that will out-compete us in every way and may just decide that we are in their way?
One answer to this (favored by Ray Kurzweil) is that we will not build robotic AI, we will become robotic AI. We will merge with our machines and they will be us. An interesting, if completely speculative, idea.
It’s all too far in the future for any confident predictions. But I do believe we should think very seriously every step of the way as we build more and more intelligent robots that are more and more automated and independent.
Klaatu barada nikto!
52 Responses to “I for one welcome our new robotic overlords.”
Leave a Reply
You must be logged in to post a comment.
Perhaps I am too fatalistic or too complacent but, like worrying about the possibility of someone stealing my identity, I cannot seem to raise my concern level about “killer robots/androids/etc” above “fun to discuss over a beer”.
Historically, the general population have always been threatened by the acquisition of “power” by an “group” willing to use it to oppress. Fortunately, like most of our “all to human” failings, the application is rare. We do live in a world, at least in the “West” (the only geography of my direct knowledge), where we live our days without being mugged hourly, having our property taken every 6.5 minutes etc.
Perhaps AI warrior robots are in our future. If they are they will certainly be misused, break down, malfunction etc. With the same certainty the majority of the worlds population will notice little impact on their daily routines.
Last point doesn’t barada have 2 d’s.
“automated military robots”… so like a cruise missile then Noel?
Of course if the machines do go bad, the only solution will be to deploy our own cybernetic organism – Kevin Warwick – to combat them.
I did my undergraduate degree in Artificial Intelligence, and I have to say that I am getting absolutely fed up of media-whoring “experts” like Sharkey and Warwick, who’ve published no decent peer-reviewed research in years, and resort to making crappy predictions like this as a way of drumming up sales for their books , TV shows, et cetera, at the expense of potentially harming their fields.
Case in point – Kevin Warwick’s prediction that we would have human-level AI in 10 years. Made in 1997 (http://layscience.net/?q=node/16).
Machines ‘To Match Man’ by April 22nd, 2029 at About Noon…
…
It has already happened.
http://blog.wired.com/defense/2007/10/robot-cannon-ki.html
No offense Steven, but I think using played dead horse memes really detracts from your work.
For pete’s sake the overlords thing was used just this week in the rogues gallery. And 900 million times online in the past six years.
Good point Niobe.
I have a pertinent video on this exact subject!
http://www.youtube.com/watch?v=oHg5SJYRHA0
I’m with you 100 percent, Dr. Novella. I’d just like to add that we should also immediately sign a treaty preventing these killer robots from travelling back in time to assassinate the mothers/childhood selves of future resistance leaders. Also, let’s be sure to keep an eye on Arnold Schwarzenegger.
Wow. Tough crowd.
We already have those, even if not too intelligent, for a long time: http://en.wikipedia.org/wiki/Land_mine
The only difference is that it doesn’t have a human-like face and/or body.
” there is one line I recommend we never cross – developing truly artificially intelligent, automated, mobile military robots.”
Not to worry. There will never be any truly intelligent robots. I could imagine vehicles that don’t require a driver, that run around targetting humans. But they could never determine which humans are enemies.
Military technology has already made our long-term survival unlikely and yes it will continue to evolve into ever more deadly forms. But your science fictions scenarios WILL NOT EVER HAPPEN.
There will never be robots with human-like bodies, and certainly NEVER with human-like minds.
And furthermore — every line that it’s possible to cross will be crossed. No one can decide for the whole world that a line should not be crossed. Unless you’re imagining an all-powerful world dictatorship consisting of scientists like yourself.
pec wrote: “There will never be robots with human-like bodies, and certainly NEVER with human-like minds.”
Just a reminder, pec was the one who criticized me (falsely) for being too dogmatic and certain in my conclusions.
I try hard to remember never to say never when it comes to technology (barring a violation of basic laws of physics). Not ever is a really long time. Of course, pec’s unstated major premise is that AI is impossible on metaphysical, not technological, grounds. A premise that has not been established.
You might consider the similar circumstances that are already occurring. That would be the “programming” of human beings into killing machines during military training and/or other circumstances.
Most humans do have “fail-safe” mechanisms that prevent/limit the killing of other humans. The military has spent a great deal of effort trying to understand these “fail-safe” mechanisms and how to over ride them. In some cases they have succeeded. Without knowing the “source code” of that fail-safe programming, the “updates” often are quite “buggy” resulting in undesired “glitches”, such as rape and murder of civilians, suicide, fragging, post traumatic stress disorder. There is no effort expended toward removing these “programming updates” when the tour of duty is over resulting in veterans who lack the “fail-safes” they had before they entered the military. This shows up as veterans who are profoundly changed by war (usually not for the better) and exhibit increased domestic violence, crime, murder and suicide. A common “treatment” that such individuals receive is many years of confinement under harsh conditions in prison, which often only exacerbates such behaviors.
Some things that are known to invoke the violence subroutines in humans are mediated through what is known as the “cycle of violence”, where individuals exposed to violence, abuse and bullying in utero, as infants, as children, as adolescents, and as adults become more violent themselves. I suspect that many of these epigenetic programming events are mediated through the low NO that severe stress produces. Many social behaviors are mediated through neural networks that have effects mediated through NO, such as oxytocin sensitive neurons. Testosterone tends to reduce the NO level in the brain, estrogen tends to increase it.
I suspect these are adaptive “features”. When one grows up in a violent and abusive environment, there may be survival and reproductive advantages to be violent and abusive oneself. In a violent world, the individual who escalates first has an advantage. In a cooperative world, cooperation has more advantages than does violence.
I think that evidence would suggest that violence does not produce a permanent solution to violence, but only one that persists until the generation that grows up under that violence matures and takes the violence to the “next level”. Unfortunately there are not very many levels between street violence and nuclear and biological weapons.
Pec’s unstated premise is that if, “our memories are primarily for telling ourselves stories about our past” (as someone recently opined), then replicating that kind of memory system in robots would lead to the same predictive problems that human have.
And he is correct for once.
I suspect he sees that a robot with an ego, with fears of injury and death, with self-confidence problems, with potential distrust of other robots built in, with hunger, and sexual desires, with empathy and a moral sense, and with a propensity for religious worship and a belief in a purposeful universe that he will also be programmed to try to ignore (inspiring the first artificial paradox perhaps), can’t be built. And if so, he is right again – we can never build a thing that will never work as intended, and when the “unstated premise” of its builders is in fact to build a robot with a mind that is fundamentally different from ours.
Simple solution, when we build our killbots we just need to build in a pre-set kill limit, then we can always beat them by sending in wave after wave of soldiers. Problem solved!
“There will never be robots with human-like bodies”
*points to Keanu Reeves*
“I suspect he sees that a robot with an ego, with fears of injury and death, with self-confidence problems, ….”
No, I simply do not believe it is possible to create a machine that does anything beyond what it is programmed to do. Our brains are machines, our minds are something more.
Yes, that is a dogmatic statement and I can’t prove it. But you can’t prove yours either. We can agree to disagree about the science fiction scenarios.
Since we cannot, and in my opinion, never will, create machines that learn and become conscious actors, all of our machines will have to be programmed, explicitly, in detail. Just like computers are now. There is absolutely no way to program all the necessarily possibilities needed to simulate intelligence.
Most of our behavior is programmed and automatic, and we resemble computers in that. Things we have already learned become automatic habits, so our conscious minds are free to continue learning new things.
Our conscious minds have some degree of freedom. This is not, as materialists insist, an illusion. It is real.
Pec, you either can’t read to the end of a sentence, or understand it when you do, or both for the same reason. What I was saying of course was you may believe that a robot with an ego, etc., CAN’T be built. You of course reply in effect that you are more simple minded than that.
Even when you appear to be right, we soon discover it’s always for the wrong reasons.
I’m not worried about the automated robotic warfare at the moment and I hope I don’t have to be in my lifetime. My concern that is somewhat related is weaponizing space. That seems to be closer than the robotic overlord “Cylon Senerio.” Glitches from solar radiation or just failing satellite equipment is enough to worry me. I’m not sweating over it but the possibility is truly frightening. Hostile foreign relations and increasing ability to get to space is making it a perfect staging point for war. How much faster can we deploy missiles without friction and newton getting in the way of a speedy launch?
Pec asks what good is intelligence when you limit its freedom, and I ask pec, what does freedom feel like to an entity with limited intelligence?
I am much more worried that a human in charge of nuclear weapons will develop a “glitch”, where that human decides that God has told him to use those nuclear weapons to destroy God’s enemies and bring on the Apocalypse.
Could a robot be made to believe that its human creators were not gods? Would it always suspect that the gods were simply lying to it?
In a society of robots, what percentage would need to exist in a state of quiet desperation?
Could a robot be expected to pontificate?
Actually the last thing we need is a set of materialist robots that have the capability to evolve.
Will the robot be programmed to act on a set of principles or to just use them as post-decision justification?
Should a robot have an enhanced sense of the ridiculous? Should its introspective abilities be as equally limiting as ours? Will there be a dominance hierarchy with a place for the pec prototype somewhere in that order?
Last and not least, can a robot say never say never without saying never say never say never?
Roy, in reply:
Yes
Probably
More data required
Yes
Agreed
No
No
No
Maybe
No
“Pec asks what good is intelligence when you limit its freedom, and I ask pec, what does freedom feel like to an entity with limited intelligence?”
I can’t imagine Roy, maybe you can tell us.
Simon: Your answers have been graded and the test has been passed to the extent that any response to your efforts is expected to be non-violent.
Pec: You mean it feels a lot like a severely limited imagination? Hey, good guess. But you should nevertheless be wary of a violent response down the road as a matter of principle. Oh, and never ask why, the answers would be beyond you.
Which reminds me, will robots need to knock sense into each others heads?
I’m told pec wants to quick start the robot educational system by making all disciplines follow a certification process based on nothing more than multiple choice exams – a certificate of attendance may also become acceptable if it proves out that all robots think alike.
why would YOU worry about automated robots?
that is EXACTLY what you proud materialists think people are!
hypocrite.
Gregory, us materialists certainly are concerned about the actions of humans, so where is the hypocrisy?
I expect suicide bombers of all persuations to join together in a massive strike against those jobs-stealing killer robots !
Hey Gregory, we think robots are people who have to have gods to do their thinking for them.
Someone should tell the ID proponents that humans are the evolutionary link between apes and robots!
Some interesting pondering about sentient robots (who love Shakespeare) can be found in Dan Simmons two book series Ilium and Olympus.
We all know what happened when ED-209 suffered a glitch. Poor Kenny. lol
pec wrote:
Er, ever heard of IFF? Identification, friend or foe. We’ve had this technology since WWII.
I’m sure your argument will be, “But that will just reduce the chances of friendly fire!” Yeah, well, war is hell.
Pec’s responses can only be predicted by considering he (or it) is actually a primitive form of computer simulated semi-logical response bot. And seriously in need of an upgrade.
Which reminds me, can a robot be degraded?
If pec uses M$ windows, does he believe that all of the crashes are programmed in? If machines can only do what they are programmed to do, then that must be pec’s explanation for why all the M$ products have such large files, to store all the pre-determined conditions under which the program will crash.
Thus a robot should never be asked if it’s mother wears army shoes.
@ deadalus2us: I’m afraid pec will call you out on this one. Currently machines can only do what they’re programmed to. That is, they respond exactly as designed, however poor the design. Even if we introduce unpredictable variables (e.g. values derived from radioactive decay, or unhandled exceptions stemming from poorly designed driver software), the computer still responds to those variables exactly how it was programmed to.
But the other side of this coin is the high probability that humans are also “programmed” to respond predictably, even to unpredictable variables. That is, determinism does not imply predictability at the macro level; or rather, it doesn’t preclude (theoretically) predictable responses to random input.
Humans are programmed to allow experience to upgrade their initial programming. Some allow for more experience than others of course.
Sharkey really needs to clear up what the hell he is talking about. Mixing futuristic hypothetical situations and terrorists taking over remote controlled vehicles really confuses the issue. In particular if one raises the valid point that bad guys could take over a current technology military machine, it is then false to extrapolate this the advanced AI taking over the world.
To really understand this area I think one needs to have engineering, robotics, computing and cognitive science under your belt because evaluating the concept of SF robotics from any other position makes the whole thing appear a lot more feasible than it really is.
IMHO and with many years of experience in all the above areas these are some issues that need addressing and why I think the SF ideal of a killer robot is a long, long way off.
1 Engineering considerations.
a)We need a machine than can move through any environment, without getting jammed, fall over, handle steps and deal with water. After 50 years of work we still don’t have one. Many can handle one environment or challenge but none have the flexibility.
b)Toughness. Machines break too easily. Sensors are too fragile (and too dumb). Human perception totally whips machine perception by many order of magnitudes. Vision systems are still crap in the real world after many decades of work. They always work less well than predicted and degrade over time.
c)Fixability. Machines still need humans to maintain them. There are zero examples of any machines with self repair. The best we have is machines with redundancy in hardware and the ability for *us* to change the program that control them. Fixing complex machines in the real messing world is a complex and finicky business that needs creativity and tactile dexterity. No machines have this yet.
d)Power source. There are no machines with the ability to recharge/refuel unless in a very constrained and deliberately designed environment. The real world is not like this. Even our cars, one of the most common complex machines that uses standard fuel openings and gas nozzles does not have automatic refueling. Current on board energy store in any robot either electric or fuel is crap. e.g. Asimo is 40 minutes.
2 Computing limitations.
a)Anything half smart is going to need a computer that is as powerful as our most powerful to date. These days computers are close to the basic needs however see below.
b)This computer needs to be really small, it needs to be low power and have a feasible cooling system for the waste heat. Go to your local supercomputer and check out it’s power requirements and AC system. To work in a human made world in needs to be no larger than human sized, especially if it needs to hunt humans.
c)As above it needs to be robust and fixable. Even if components can be replaced you need a machine that can do that. Currently there is no sign on the horizon on a machine than can do repairs on computers out in the field, apart from humans.
3 Software/ Cognition
a) It’s extremely difficult.Well D’Oh! After 50 years we have come exactly how far with AI? It’s a joke. Lot’s of money, lots of time, lots of really clever people. This indicates it is a major problem with no quick fix. This is not one of those incremental getting better ideas like solar cells , better artificial skin or even fusion power. We could well be missing a fundamental understanding about intelligence. (Big hint: Animals use *massively* parallel analog ‘computers’, computers use digital and not connected in the same messy way. )
b)Memory. We don’t even know how human memory works very well yet although we are getting closer. We need to solve how new memories form, how conflicting memories are resolved and how memories are forgotten. We have to work out what memories to keep and what to not bother with. This is a major problem. Unlike what Brooks may think, having a memory is vital so you can not repeat your stupid behavioral mistakes.
c) Learning.How to learn skills, how to learn what is important to know about an environment. How to apply one field of knowledge to another. How to improvise with other tools when the tool you want is not available. How to teach others what you have learnt. Humans 1+ : AI 0
d) Decision making. We need machines that are not rational but use rough and ready rules because these heuristics are robust and handle noise and variation in the environment. To the best of my knowledge no one is working on this sort of AI. Traditional rational decision making and algorithms might seem fine in an AI degree but are generally useless for the real world where you need robust but not brittle decision making processes. It needs a system that can resolve conflicting decisions and come up with the best long term solution for the greater good of whatever motivates it.
e)Understanding other intelligent beings. Human or otherwise. To understand the environment you need to be able to ‘be in someone else’s shoes’, to work out what they are thinking and to be able to infer this by observing their behavior and listening to them. Just reacting to them without building up a model of what they are thinking is not very good if you are an uber-robot trying to take over the world. You will be out-smarted and tricked.
I could go on and just about write a book about everything that is needed, there is simply so much and above is the off the top of head ideas.
Furthermore all of the above problems need resolving first and then incorporating into one package. However this means you you get a non-mobile AI before other things and that is an issue you need to deal with first. Having a software based AI with the ability to evolve it’s “thinking” millions of times faster than humans ever did is an issue to address way before a) it happens or b) it does and bothers with walking/crawling/flying around.
And as for the near term ,say 30 years, bad guys with GPSr guided RC planes with bombs, homebuilt rockets, suicide gas tankers, yeah whatever. I’ll worry about cancer and heart disease, they are a thousand times more likely to kill me. It’s much easier to kill people you hate with biological weapons or accidentally because you screw up your nuclear arsenal safety protocols. Robot overlords? I tremble in my boots…not. Before we even think about robots we need to be a lot further down the track with AI and currently the wheel’s still turning but the hamsters dead on that one.
That was a keeper!!
Is it not already happening?
look to South Korea
http://abcnews.go.com/Technology/story?id=2504508
Well said, Zytheran… I like the SGU and all, but the parts where robots and AI are over-applauded uncritically are my least favorite parts, he.
When you’re basically NOWHERE in a certain field, it’s not really good practice to extrapolate so wildly that it becomes ridiculous.