Yesterday, I wrote about the mystery of why spam was so bad at being spam and I claimed that it was a mystery that seemingly defied explanation. None of what I proposed as possible answers was really satisfying. In order to answer this question, I think you have to look further afield and ask some other interesting questions: “Why has there not been a non-pathetic foreign terrorist attempt on US soil since 9/11?” and “Why has there only been a handful of truly crippling computer viruses in the last 10 years”

Our first instinct is that such occurrences are rare because they are difficult. However, neither of these tasks actually are difficult. Two guys in a van managed to terrorize Washington DC for a month and no amount of security precautions could have prevented them from doing so. The Sasser worm was written “by someone that could barely get the code working” and attacked a security flaw that had been noted and patched months ago and other worms haven’t been much more sophisticated. Such things are not trivial but they aren’t of such herculean difficulty that would be sufficient to explain their rarity. Just why exactly isn’t there a legion of evil geniuses who are routinely executing the downfall of society?

An evil genius is anyone who is both a genius and evil where “Evil” encompasses everything from trolling to keying someone’s car to pedophilia, “Genius” is anything which evokes any degree of “huh, why didn’t I think of that?” or “That’s clever”. As a rough approximation, we assume that the number of evil geniuses can be calculated by multiplying the proportion of people who are geniuses with the proportion of people who are evil. But what I’ve noticed through looking at a huge range of diverse social systems is that evil geniuses exist at a stunningly lower frequency than this naive calculation would have us believe. The number of evil geniuses is so off base from the naive calculation that it indicates a our model of the world with regards to evil geniuses is unsalvagable and needs to be replaced, not just tweaked.

Such a claim has radical implications for the design of social systems as so much of our thinking about security, about design and about society is obsessed with preventing evil geniuses from wreaking havoc that we don’t even stop to notice that they aren’t.

Part of the reason we’re so obsessed with evil geniuses is because we think we know what they’re like: they’re just like us except they actually do the evil things we think about. Bruce Schneier, one of the most widely read security experts in the world writes about how

Uncle Milton Industries has been selling ant farms to children since 1956. Some years ago, I remember opening one up with a friend. There were no actual ants included in the box. Instead, there was a card that you filled in with your address, and the company would mail you some ants. My friend expressed surprise that you could get ants sent to you in the mail.

I replied: “What’s really interesting is that these people will send a tube of live ants to anyone you tell them to.”

- The Security Mindset

“Why golly”, the man with the Security Mindset says, “I’ve found a great way to exploit this system. It’s lucky I’m a good person because all that is stopping me from executing this exploit for my personal gain is my innate goodness.”

It’s easy to imagine a person who is just like me except without my innate goodness. As a result, it’s easy to design a system with defenses against such a mythical attacker. What we completely fail to notice is that, most of the time, such an attacker simply does not materialize. But even though evil geniuses might not be a major problem, evil behavior most definitely is and it’s in our best interests to design a system which is resilient to pathological actions such as trolling, flaming and abuse.

Our naive view of the world is that we mentally segment people out into “good people” and “bad people”. Good people are people like us and bad people are people like us, except without any morality. The work of Milgram and Zimbardo shows though that goodness is largely a property of circumstance and the more correct way of thinking about the world is that most people are ordinary people and there are good situations and bad situations. If evil people are inherently evil, then it’s easy to imagine an evil genius. However, if evil is a product of the situation, then maybe the reason there are no evil geniuses was because noone gave them permission to be evil geniuses. The reason why Milgram and and Zimbardo managed to cause people to become evil was by relying on authority to signal that such actions were permissible. Genius, by definition, cannot provide be provided such social proof because you’re doing something new and unexpected. Without such social proof, it’s very hard to create an evil situation and, as a result, evil genius is hard to come by.

Such a statement has radical implications for design: you can cause pathological behavior simply by putting in visible mechanisms to prevent pathological behavior. We look to social cues within the system to understand acceptable bounds of behavior and in certain cases, one could reason that if the designer spent so much time building safeguards against certain behaviors into the system, such behavior must be prevalent and thus, acceptable to experiment with. In some cases, the correct approach to obsessing about the security of a system is to leave the system deliberately unsecured so that it does not even occur to people to test the security.

The “No Evil Geniuses” hypothesis is a radically different way to think about the world and one I don’t even think I can completely justify. At the same time, after having looked at all of these disparate cases in which there simply isn’t any other good explaination, it’s one I’ve been increasingly forced to take. Whenever I’ve gone out on a hunt to spot a rich treasure trove of evil geniuses, I’ve never been able to find them. Maybe there’s a simpler, more coherent explaination for all of this but until I find it, I’m going to bill this the No Evil Geniuses Paradox.

Related Posts

  1. A real case of an evil genius
  2. Facebook credits: Brilliant, Evil or Brilliantly Evil?
  3. The no obnoxious rich people paradox
  4. Nov 11th (day 29): Bumblebees and Spam
  5. Provably Unsolvable Security
  • _
    Mitnick?
  • A related question is why there seem to be so few evil geniuses among existing conspiracies, like national security agencies, organized crime, militaries, "terrorists" etc.

    They can afford to hire geniuses. They have goals that are recognized as "evil" by many people outside their organization. They don't have that many moral qualms - they've all been known to kill random people if it suited their interests. So why don't they use evil geniuses more?

    Or perhaps they do, except they're so good it almost always stays secret. For example if CIA was responsible for protests in Iran, that would be an evil genius success, and they would obviously not want to talk about it.

    I think the problem is open.
  • Hang
    I don't think they being overly competent is the correct explanation. It strikes me as far beyond the capabilities of any organization to pull that off consistently.

    I think your observation is an interesting avenue of speculation and I suspect the correct answer has to do with one or more of your assumptions not being valid.

    One thing to consider is that almost no organization regards *themselves* as evil. Terrorists think of themselves as freedom fighters, security agents think of themselves as patriots.
blog comments powered by Disqus