I’ve been reading Why should the boss listen to you by James Lukaszewski and it’s one of those books which is so obviously right that it shouldn’t need to exist. One of the very obvious things that the book says is:

you should be helpful to your boss

Simple, trivial and easily accomplished right? It’s not as easy as you think.

The second most useless form of help

When I was writing my PhD and I met another academic, we would talk shop and I would explain my research. More often than not, they would suggest a paper to read, a new avenue to explore or a new perspective on the issue. Of course, they were just trying to be helpful but the majority of the time, these conversations were the second most useless form of help I could have gotten.

I am willing to wager that there has not been a PhD in history who thought he did not have enough papers to read or that their project was too tightly scoped. Working on a PhD program is like trying to empty the ocean with a tablespoon, the list of potential work would occupy several hundred lifetimes. Given that you only have 4 – 10 years to complete a PhD, you are forced to prioritize and intelligently ignore everything that is not within your immediate scope.

Every PhD student has an internal priority list of papers they could feasibly read, papers they would love to read and papers they know to be useful but they have no hope of every getting around to reading. So when you propose a new paper that is only tangentially related to their topic, they’re just going to smile politely, let you email it to them and it’s going to slide to the very bottom of their to-do list. This is the second most useless form of help.

The most useless form of help

The only useful forms of help

iPhone applications provide an unwitting peek at what companies believe is the real purpose of their product. Because of the limited form factor of the iPhone, developers are forced to release cut down versions of apps and what features they choose to include tells a lot about their priorities.

For example, the youtube iPhone app does not even suggest that comments exist wheras it devotes significant space to “similar videos”. Clearly, youtube believes their business success lies in encouraging people to browse through more videos. This is reflected in their benign neglect of the web based commenting system which I think has the potential to be one of the more interesting arenas on the web if it were handled well.

The facebook iPhone app features prominently the ability to take pictures and upload them directly to your profile as well as a well integrated commenting feature. Chat as well is featured heavily in the facebook app. Curiously enough, events are completely missing from the app (although not the web interface) which I regard as a major limitation. Also missing is the ability to forward messages or do anything other than a straight reply. Facebook mail has the potential to become the de facto email replacement but it seems relatively clear that this is an area that Facebook is not super interested in as there have only been a trickle of new functionality related to it’s messaging service.

Google maps allows you to plot a route via car but it doesn’t include the walking or public transit modes that it has on it’s website. Even more annoyingly, it doesn’t have the ability to dynamically change your route to a location as you move around which makes it effectively useless as a GPS navigation system.

Some of these insights would have been obvious without looking at the iPhone app version of websites but I think it does place these distinctions in starker contrast. It doesn’t just have to be the iPhone either, looking at any reduced functionality version of a site can give you a sense of what their developers regarded as essential and optional.

Mike Burns has some interesting thoughts on how non-testing code is trivial and it lead me to thinking about placing testing above development in the corporate heirarchy. In a normal organization, people start off in testing and maintenence and then only move up to development once they’ve proven their chops.

What I am envisioning is that everybody starts out as a developer and the development work is considered trivial and mindless because it’s aimed solely at passing tests. Once you’ve proven yourself as a good developer, you get promoted into testing. Once you start testing, you can’t ever write code again on that project and the job of the testers is to make the developer’s lives hell by making sure the code base is rock solid.

This has a number of nice features that I can see: First of all, it replicates the common “hazing” pattern that we know reinforces group solidarity. The job of the senior members is to make the lives of junior members horrible and junior members put up with it because they know some day they will get to do the same. (As an aside, there’s an interesting educational theory which says that bullying is a product of an age segregated education system. Before mass public education, children mainly interacted in mixed age groups with status based on age which was far more egalatarian because everyone at some point got to be the alpha of a group)

By forbidding testers and developers to communicate except through a chinese wall, it encourages both groups to form a unique culture and language which further enhances solidarity and cohesion. Additionally, the develop–>test heirarchy naturally encourages an evolution into develop –> test –> design which can allow for a much more organic growth into a design oriented company.

Of course, there’s a few downsides to this management structure as well: Development inherently requires more technical skill than testing and so it’s an inefficient use of expertise as those most competent at developing will be siphoned off. Additionally, not everyone is equally adept at both soft skills and hard skills and having a promotion path that moves from hardcore technical skills to people skills is only suitable for some people.

Say you’re a hardcore algorithms guy who loves working on scalability problems. You don’t want to move up to testing, testing is boring to you. Whats more, even if we let you become a senior developer, you’re going to get frustrated because all the smart programmers under you are going to move on.

Test driven management seems like an interesting way to structure a company and I’d be interested in hearing examples of people who are using something similar to this sort of approach.

It’s been noted before by many people that programming should be thought of as craftwork, not manufacturing. The economics of programming is such that the cost for each incremental unit is 0, so it makes sense to view development as a source of revenue generation and not a cost to be contained.

The most visible manifestation of that seems to be the Google style perks that seem to be de rigeur now among web 2.0 companies. Basically, the equation boils down to more perks means happier programmers and happier programmers mean the most number of feature points per hour.

But if programming is artisanship, then I think creature comforts are merely the most rudimentary of what we could be doing and it’s instructive to look at true artisanship in other field. In particular, when we look at examples of the truly haute in couture or cuisine, one of the things which is most striking is that they revel in their stunning inefficiency and that this is what makes them special. There is no attempt to streamline or rationalize or redesign for efficiency purposes. If a suit jacket can be made even incrementally better by individually hand stitching each seam so that the stripes are aligned, then this will be done as a matter of course. Sure, you could get a suit for half the price with a barely indistinguishable drop in quality but this doesn’t matter because what artisans are striving for is not efficiency but quality.

So what does this mean for programmers and programming shops? What would happen if we abandoned efficiency as the primary goal of the organisation? What stunningly inefficient paradigms could we embrace?

The first that comes to mind is pair programming. Maybe pair programming works, maybe it doesn’t, I don’t know. I’ve not seen that many people talk about it outside of the hardcore evangelical XP community and it’s easy to see why. No matter how you manage to slice and dice it, two developers working on a single machine are just not going to be able to produce as much code as both of them working independantly and XPers trying to sell you that it does start stretching for increasingly absurd arguments. To any organisation, the idea of pair programming sounds like an extraordinary extravagance which would be impossible to justify under any budget.

Another thing which seems curiously missing is the concept of a “stage” or sabbatical or any other form of dedicated educational experience. In haute cuisine, chefs will stage at each other’s restaurants all the time to pick up new skills or gain different perspectives on food. Why is it that no company offers a 6 month, paid time off so that their employee can go work at Google or Facebook or any other place doing interesting things? What would happen if people were regularly allowed to drop out for a few months at a time to do pure research?

Even in the most enlightened companies right now, programming is still viewed as a fundamentally productive activity (in the sense that the goal of it is to produce things). It’s a very tough sell to propose anything which doesn’t demonstrate an increase in function points/hour/dollar. But if we truly believe programming to be an artisanship activity, then we should consider optimising for something else, optimising for quality so that even the most extravagently inefficient touches are celebrated because they make the product better. Could this work economically as a model for commercial programming? I honestly don’t know yet…

Discovered a rather unpleasant surprise. I assumed Rails Rumble was ending at midnight PDT, turns out it was actually midnight GMT which meant we were scrambling over the last 8 hours to get everything done. Anyway, a somewhat function version of the site is up at:

http://dontsendit.r08.railsrumble.com/

Site is definately very barebones and lacking in decent design/features at the moment.

We’re spending the weekend at bumblebeelabs working on a project called dontsendit as part of the Rails Rumble. The concept of dontsendit is very simple: what if we could write emails and set conditions so that they are sent only if you fail to do something? How would that change the dynamics and nuance of email?

As a concept, dontsendit is fairly technically trivial. There’s only 3 major pages and it shouldn’t take more than a few lines of code. Where the real challenge has been for us is that, while it’s easy for us to tell people how it works, it’s been tough to explain what it can do for them. There’s a major cognitive leap from “I have a problem” to “dontsendit is the perfect solution for it” and practically noone can make that leap unaided.

The problem is, dontsendit could be used for potentially quite a number of different things. Some of them more compelling than others. One of the tricky design arguments we’ve been having is whether to go for general but bland (aka: the swiss army knife approach) or specific but opinionated (the scalpel approach). There’s much to be said for both sides which is what makes it a perennial topic for debate but my gut instinct is that general but bland sounds seductive at first but will ultimately fail.

Anyway, keep an eye out for dontsendit.com, the final version should be launching by midnight tomorrow. I wouldn’t try testing it before then unless you want your emails to be publically viewable. It’s still very much a work in progress.

Personal growth is never a smooth process. There are many potential pitfalls along the way which can trap people for years at a time until they see the neccesary advanced wisdom to unstick themselves. One of these that I’ve been thinking a lot about recently is a concept I call “you can’t get here from here”.

Let me give you an example: Of all the most deeply spiritual people that I know, all of them have had some sort of deep crisis of faith where that have had to grapple with serious doubt and all of them have confessed to me that such struggle has been a key step in deepening their faith. I would go so far as to argue that such a crisis of faith is a neccesary step to forming a truely genuine bond of faith as it forces you to seriously grapple with your existential doubt rather than simply ignoring it. But if this is true, if a crisis of faith is a neccesary component, then you simply can’t get to having strong faith simply by having strong faith. You can’t get here from here.

You can’t get here from here is a pernicious process because the path to the goal inherently requires a leap of faith. None of my friends who went through a crisis of faith were thinking “oh good, I’m having a crisis of faith now which is Step 8 in my Good Christian handbook”, they were too busy freaking out that what they had taken for granted all their lives was being pulled out from under them. And indeed, none of them knew what they would look like coming out the other side. For some it grounded their faith, for others, it was the first steps towards atheism and for a few, it simply left them deeply confused and hurting. But for all of them, it required bravery to fully commit to the path they were on rather than staying in a state of denial. How many others were there who came on the verge of a crisis of faith but resisted because they deeply believed that the road to more faith was more faith?

I’m talking about others now because “you can’t get here from here” is a deeply personal process and it’s possible for me to see it more objectively from the outside. Examples from my own life are always messy and confusing and full of doubt but I think there are some I’m willing to tentatively put on this list:

• You can’t become truly arrogant until you’ve become humble
• I used to think being nice was about saying nice things. Then I realized that being truly nice is to think nice thoughts and to do this, you must say your nasty thoughts
• In order to master a sport to the level of being able to analyze constantly, you have to at some point, stop thinking
• You can’t be truly generous until you’ve achieve selfishness

If you don’t understand what I mean by these, they might be advanced wisdom. On the other hand, they could just be bullshit.

I was talking to a friend the other day about the nature of argument and how to argue well and it occured to me that I needed to take a step back and convince him that it was possible to argue well. Argument just seems like one of those things you do. You sit and you talk and you generally say the first thing that comes into your head. That argument would require training and skill is something which doesn’t appear immediately obvious at first grasp to people.

We seem to split up the list of personal qualities into skilled and intuitive. In the skilled category would be things building a house, playing a game of chess, arguing a case in court or solving math problems. On the intuitive side, we have things like having a good sense of humor, being co-ordinated, having the ability to draw well or coming up with good ideas.

The difference between the two is that we believe that to be good at skilled things is a process of mastery of certain skills which is relatively unmysterious. On the other hand, being good at intuitive things is something wholly mysterious that seems largely innate.

What’s interesting about argument is that it seems to have shifted from a skilled task to an intuitive one. In medieval times, rhetoric was part of the trivium, along with grammar and logic. People would spend 1/3rd of their university education solely on learning the art of arguing and people understood instinctively back then that you had to work to become a good arguer. Nowadays, rhetoric is a minor part of a minor department and argument seems like a purely intuitive process.

It seems to me a plausible explanation is that we rely on social signals to cue us in on what is skilled or not. Because we see people spend years learning to be a professional doctor but not learning to be professional arguers, we tend to believe one requires skill and the other does not.

So why am I going on about this? Because such a heuristic is imperfect at best and being aware of how it is flawed can help you gain an incredibly easy win over other people. By correctly identifying something as requiring skill, you can start to gain immediate improvements and, what’s more, others will believe that this improvement is innate and both be impressed with how good you are and not try and compete with you.

Amazon Prime has totally spoiled me. I ordered the book The Pixar Touch on a whim two days ago and it arrived this afternoon. I’d always deeply admired Pixar and this is a sentiment many others similarly share. Pixar, just the word seems to be imbued by a touch of magic and the company itself seems blessed, if not by God, then at least by some ineffable air of genius.

Deciding to do a bit of light skimming tonight, I dove into the book and didn’t put it down until the last page. By dissecting and laying out the Pixar story, David Price brings to life the human, behind the scenes elements to the seemingly larger than life success story. Like any retrospective, the history is gentlely massaged and bits are snippet to form some core of a story but even looking past this, I learned some surprising lessons.

At Bumblebee Labs, one of the things I’ve instinctively felt is that [i]soul[/i] is one of your most precious and yet most luxurious assets at the early stages. That for a company to be truely successful, not only from a marketplace perspective but also from personal accomplishment, it needs to be infused with a deep and consistent sense of personality and purpose. At the same time, it’s often justifiable criticism that spending the early stages focusing on establishing the “philosophy” and “branding” of a company is just so much mastabatory wank that is a more pleasant alternative to getting more work done.

Few people realise just how young management is as a discipline. The roots of the idea that companies required some sort of systematic, formalised process for directing the course of operations only really took root during the American railroad boom and the need for communication and synchronisation across large distances. Looking back just twenty years some things were back then and give you a sense of how management could change twenty years from now. Just being aware of this velocity of change can keep you aware and on the lookout for some of the genuinely new and lasting innovations which are coming down the pike.

Ultimately, though, the story of Pixar is about the story of talent and that’s something which can never be learned just from a book. At some point, each and every one of us has to face up to the fact that no amount of planning or luck or preparation can make up for the need to be simply good at what you do.

I’m in love with statistics. Knowing statistics has changed how I view the world and it’s often hard for me to convey this to people because statistics has been tragically misrepresented to the public. Most people think that statistics is a subset of math but I believe that, fundamentally, statistics is a philosophy class and I wager that if it were sold as that, it would be much more popular.

At it’s core, statistics is an epistimology (the philosophy of knowledge) that happens to use math as it’s language. It’s about probing the nature of certainty and doubt, understanding the power of knowing and the limits of knowledge.

Let me give a simple example: Your friend has a coin which is either fairly weighted or weighted to land Heads 80% of the time. You observe a series of coin tosses and it comes down HTHHHTHTHHTTTHTHTHHHH. What does this tell you about either hypothesis? What does this new knowledge now allow you to infer about the nature of the world around you? Notice that certainty is impossible, no matter what sequence of coin tosses you observe, it’s possible for it to be generated by either hypothesis. The knowledge you are gaining is inherently probabilistic, inherently statistical.

How does each additional coin toss influence your beliefs? How many coin tosses are required for you to have any useful knowledge? If you have less than that number, what is the nature of your belief? All of these questions are deeply philosophical but they cannot be answered without an analytical toolkit.

Understanding statistics rewired my brain, made my see everything in the world around me in a different light. It was a mental augmentation that made me a quantum leap smarter. But I’m not going to lie, statistics also kicked my ass. I rarely struggle to master anything but the first statistics class I took, I got a 68/100 and came out of it unimpressed. I came into statistics like I did any other math class and I focused on learning statistics as a skill to be mastered. And the work was challenging enough that I never thought to look for the bigger picture, to look for a mental framework to fit it all under. As a result, I could grind out the calculations and know what the result was but the understanding was not there. It wasn’t until I took statistics again in Graduate School and had some background in what I was learning that I started to see the underlying roots of statistics.

I think the way statistics is taught now has had a profoundly detrimental impact on how it has been applied. There are some that argue that the recent financial crisis is fundamentally rooted in financial quants who were only interested in applying statistical tools without being fully aware of the nature of what they were doing.

If statistics had been described to me as a philosophy class, I would have come in much more aware of the conceptual side of it rather than merely focusing on the tools and techniques. I would have understood it as a way of thinking. The problem is, you can’t at the same time divorce statistics from the math. Without the mathematical rigor, statistics is an empty husk. Philosophy majors take philosophy precisely to get away from math and Engineering/Science majors took their subjects to get away from the wishy washy abstract thinking of philosophy. It’s hard to find people who have an affinity to both and when you only have one semester to get through as much materiel as possible, covering the philosophical side is going to severely limit how deeply you can dive into the material.

Still, after speaking to a friend who revealed to me her choice of major hinged solely on not having to take a statistics course, I wonder if things had been different if she knew it was all about philosophy?