Posts Tagged ‘reality’

Anything you think is either unoriginal, wrong or both

by Hang

I first discovered this obviously wrong truth when I was doing my honors thesis. Time and again, I would come up with a novel idea or a neat algorithmic trick. Some of them, I would discover had already been invented 3, 5, sometimes 10 years before I came up with it. But the ones I was absolutely sure nobody had published before because I had scoured the literature and covered every approach. Well, all of those original ideas turned out to have some hidden, unforeseen flaw that rendered them either trivial or actively stupid. This lead me to formulate the belief that “anything you think is either unoriginal, wrong or both“. Like all obviously wrong truths, it has the paradoxical property of being obviously wrong and also true.

The premise for the statement comes from the simple observation that good ideas survive and bad ideas die. This means there exists an entire class of awful ideas that people come up with time and again only to eventually discover their wrongness and then abandon them. Every person who discovers them believes themselves to be wholly original since nothing of the sort exists in the world and each of them is met with disappointment, sometimes after many years of sweat and toil. But because failures are almost invisible, they leave no warning signs to future generations that this is an awful idea that should be avoided*.

Anything you think is either unoriginal, wrong or both” is an acknowledgment of your own stupidity. Your first instinct, when you come up with a new idea, should be to try and find out if anyone else has done it before. Your second instinct should be to try and find out if anyone’s done it before. Your third, forth and fifth instincts are to ask how come everyone else figured out this was a dumb idea and I haven’t? If you’ve gotten this far and you still haven’t discovered anything useful, you should start feeling a little bit uneasy, it probably means you weren’t smart enough to discover how wrong you are.

If you have discovered the prior art or the fatal flaw, then breathe a small sigh of relief. Unoriginal ideas are GOOD, wrong ideas are GOOD. An unoriginal but right idea is still valuable to all the other people who’ve never heard of it and chances are, if you’ve never heard of it, there will be a significant fraction of the population to which bringing this idea contributes value. Wrong ideas do more to teach you more about the world than right ideas because they teach you about some discrepancy between your expectations and the world, The corrective force of wrong ideas is what allows you to deftly cut to the core of any issue and tease out just where assumptions are weak and likely to fail.

But if you’re lucky, over the course of your life, you’re going to stumble across many ideas which are both original and right, in which case it’s still better to treat them as unoriginal and wrong. Believing an idea is unoriginal and wrong makes that idea do more work. You attack it more fiercely and from more angles. You keep on asking people if the idea sounds familiar and you’re eager to seek feedback because you’re so damn curious to discover why it could be so wrong yet elude you for so long. In doing so, you disassociate the idea from your ego so that you can take criticism about it calmly and dispassionately. Eventually, that drive of curiosity will force you to action, just to finally prove how this idea is flawed. Treating an idea as unoriginal and wrong means that the only standard you’re willing to accept is success. This brings a clarity or purpose that cuts through the confusion when executing upon that idea. Other people may be willing to make excuses or caveats that salve their ego but, as far as you’re concerned, if an idea is not successful, it’s not right**.

Anything you think is either unoriginal, wrong or both” is an idea that also applies to itself. I’ve been slowly chewing over this idea for almost four years now and it’s been frustrating to me that so far, I haven’t been able to find someone else that’s expressed it as a similar sentiment which by de facto, makes it wrong. I’m putting this out there to invite the embarrassment of someone pointing out the obvious source or the obvious flaw that I’ve managed to miss for so long. Please, tell me how I’m stupid, it would be a welcome relief.

*Some people, when first discovering this problem, come up with elaborate schemes of recording all of these common awful ideas so that future generations can avoid them. This, unfortunately, is a common awful idea.

** not right and wrong are different concepts in the same way that not being a millionaire is different from being homeless.

January 22 2009

It’s not illegal unless you get caught

by Hang

There’s a popular misconception about the legal system which a lot of technical people fall under which regards the legal system as a set of laws and the role of lawyers and judges is to enforce those laws. Such a mental model is understandable as it maps well to what we’re used to in the world of computers. The laws are the source code, the legal system is the machine and the citizen is the user. Unfortunately, it’s not correct.

Unfortunately, a too strict interpretation of this model can lead to unfortunate misunderstandings of the law. In a practical sense, laws aren’t laws until they’re applied. That not all laws are applied consistently and universally is a feature of the system, not a bug.

Al Capone was prosecuted for tax evasion. Do you really think the tax evasion law was put into place to catch tax evaders? No, it was deliberately put in place as an overly broad law that is only ever enforced when no other charges can be brought.

How laws are enforced add’s a more nuanced and flexible layer to the legal system which allows it to adapt to the complexities of human society. The issue of enforcement is often ignored (usually to the arguer’s advantage) in legal discussions and it leads to a distorted and absurdist view of the legal system.

PS: I’ve just noticed that my last blog post was the 100th post on this blog. Go me!

Legitimate and cargo cult ideas

by Hang

There are two types of ideas which appear very similar yet behave very differently and the ability to distingush can be very useful. Legitimate ideas are not neccesarily ideas that are right but their distinguishing factor is that they have an adequate response to all developed criticism. That means one of the hallmark properties of a legitimate idea is that if you’ve heard a criticism against it, that criticism is probably flawed. What’s more, the mere attempt at trying to demolish it looks foolish and woefully ignorant. If a legitimate idea is demolished, it’ll be demolished from the top, by the people who are most intimately familiar with it. Evolution, I believe is a legitimate idea. All attempts to debunk evolution merely reveal the debunker’s lack of understanding of evolution.

Cargo cult ideas are those which have all the outward trappings of a legitimate idea but without the social process that causes those trappings. It survives not because it can survive criticism, but because it carefully prevents legitimate criticism from affecting it. Cargo cult ideas can survive for a surprisingly long time despite the presence of arguments against it because their very survival involves aping legitimate ideas so closely that it can use the same refrain: “if you’ve heard a criticism against it, that criticism is probably flawed”. Cargo cult ideas are undermined from the bottom up rather that from the top down as it involves a loss of faith in the system.

The structure of argument is so different between legitimate and cargo cult ideas that it becomes impossible to argue unless both people are on the same side. You can see this tension play out with the atheist vs Christian arguments where atheists take the side that Religion is a cargo cult idea masquarading as a legitimate one. Christians on the other hand feel safe dismissing atheist arguments because well, even though they personally don’t know the rebuttal to the argument, they’re sure some learned Christian scholar surely does. Why, this athiest simply doesn’t understand the full subtlety and intricacy of the Christian position and all they are doing is revealing their ignorance of it.

How do you distinguish between the two? There are some useful zero knowledge proofs but they’re tricky because ultimately, the goal of a cargo cult idea is to become indistinguishable. I think in order to do so, you need to learn enough about a subject to reach the cliff. The cliff is the point where a simple question, well stated will be rebuffed rather than answered. No good answer will be forthcoming. The problem with this is twofold: It’s impossible to assert that no answer can be found unless you’ve read all of the literature on the problem and it’s impossible to ever assert the cliff does not exist unless you reach the end of the field. Neither of these are practical goals so only probabilistic measures are possible.


January 5 2009

The natural grain of cynicism

by Hang

Every media type has a natural “grain”, a gentle structure that tugs and shapes the work within it. You can write a novel about your heroine that has piercing blue eyes and the audience with believe you. In a movie, to convince the audience that your heroine has piercing blue eyes requires that you go out and find an actress with piercing blue eyes and this is significantly more difficult.

The natural “grain” of movies is hucksterism. It’s easier to portray an artist who is a talentless, pretentious hack who exploits a gullible audience than it is to portray an artist with genuine talent who receives the accolades they deserve. It’s easier to show a motivational speaker who only speaks in pseudowisdom than one who says genuinely wise things.

Literature is the medium of heroes, of people who are admired by others because they do great things. Movies are the medium of dupes and cons, of the people who are admired by others despite having no discernable talent. Is it then a coincidence that the rise of the long form novel coincided with the spread of modernism with it’s exultation of progress and cinema coincided with the move towards post modernism and the admiration of aloof cynicism?

Nov 2nd (day 21): Obviously wrong truths

by Hang

When I was in my very first undergraduate programming class, they hammered into me on very important truth:

The compiler is never wrong

The compiler has no bugs in it, the libraries have no bugs. If you’re not getting the output you expect, then the bug is in your code. Nearly every week, someone would be there furiously muttering to the tutor that he just needs to LOOK at this example because the code is so OBVIOUSLY correct that it MUST be a compiler error of some kind. And every time it happened, the tutors would simply smile complacently back and remind the student that “The compiler is never wrong”. Eventually, with enough repetition, we understood this fact down deep into our bones and I think it’s made us better programmers as a result of it.

On the face of it, this is absurd. Compilers are programs just like anything else and they contain bugs like every other program. If we were talking about established, battle scarred compilers like gcc, you might be able to make a credible argument but we were working with the Glasgow Haskell Compiler which most certainly did have bugs in it.

The statement “The compiler is never wrong” has such power because it’s so patently easy to prove false. And as I grow older and think I understand more and more about the world, some of the most powerful beliefs that you can hold are the obviously wrong truths. You can never tell an obviously wrong truth to someone who is not ready to hear it because it’s so obviously wrong. You need to take a leap of faith and accept that something can be obviously wrong and still true for such things to make sense.

If this sounds supiciously like what you’ve heard religious people say, it’s because maybe this is what religion is…

Oct 14th (Day 2): Statistics is a philosophy class

by Hang

I’m in love with statistics. Knowing statistics has changed how I view the world and it’s often hard for me to convey this to people because statistics has been tragically misrepresented to the public. Most people think that statistics is a subset of math but I believe that, fundamentally, statistics is a philosophy class and I wager that if it were sold as that, it would be much more popular.

At it’s core, statistics is an epistimology (the philosophy of knowledge) that happens to use math as it’s language. It’s about probing the nature of certainty and doubt, understanding the power of knowing and the limits of knowledge.

Let me give a simple example: Your friend has a coin which is either fairly weighted or weighted to land Heads 80% of the time. You observe a series of coin tosses and it comes down HTHHHTHTHHTTTHTHTHHHH. What does this tell you about either hypothesis? What does this new knowledge now allow you to infer about the nature of the world around you? Notice that certainty is impossible, no matter what sequence of coin tosses you observe, it’s possible for it to be generated by either hypothesis. The knowledge you are gaining is inherently probabilistic, inherently statistical.

How does each additional coin toss influence your beliefs? How many coin tosses are required for you to have any useful knowledge? If you have less than that number, what is the nature of your belief? All of these questions are deeply philosophical but they cannot be answered without an analytical toolkit.

Understanding statistics rewired my brain, made my see everything in the world around me in a different light. It was a mental augmentation that made me a quantum leap smarter. But I’m not going to lie, statistics also kicked my ass. I rarely struggle to master anything but the first statistics class I took, I got a 68/100 and came out of it unimpressed. I came into statistics like I did any other math class and I focused on learning statistics as a skill to be mastered. And the work was challenging enough that I never thought to look for the bigger picture, to look for a mental framework to fit it all under. As a result, I could grind out the calculations and know what the result was but the understanding was not there. It wasn’t until I took statistics again in Graduate School and had some background in what I was learning that I started to see the underlying roots of statistics.

I think the way statistics is taught now has had a profoundly detrimental impact on how it has been applied. There are some that argue that the recent financial crisis is fundamentally rooted in financial quants who were only interested in applying statistical tools without being fully aware of the nature of what they were doing.

If statistics had been described to me as a philosophy class, I would have come in much more aware of the conceptual side of it rather than merely focusing on the tools and techniques. I would have understood it as a way of thinking. The problem is, you can’t at the same time divorce statistics from the math. Without the mathematical rigor, statistics is an empty husk. Philosophy majors take philosophy precisely to get away from math and Engineering/Science majors took their subjects to get away from the wishy washy abstract thinking of philosophy. It’s hard to find people who have an affinity to both and when you only have one semester to get through as much materiel as possible, covering the philosophical side is going to severely limit how deeply you can dive into the material.

Still, after speaking to a friend who revealed to me her choice of major hinged solely on not having to take a statistics course, I wonder if things had been different if she knew it was all about philosophy?

August 1 2008

Food miles is bullshit

by Hang

I’ve posted this before in various venues but I thought it was worth a revisit:

So there’s been a lot of talk in the last few years of “food miles”, eating locally and 100 mile diets. Some of the supposed benifits of eating locally is that you become more in tune with the seasons, you support your local community, you eat fresher food and just general all round feel goodness. Now all of these are valid claims and I am not disputing any of them. But the chief claim that the “food mile” movement is making is that eating locally helps the environment through lowering the use of oil. On the face of it, this sounds fairly intuitive but I wasn’t convinced so I decided to dig a bit further and try and answer the question does the choice to eat locally decrease the amount of carbon emitted and it seems like the answer is no.

(more…)

Copyright ©2009 BumblebeeLabs — Theme designed by Michael Amini