Posts Tagged ‘HCI’

February 12 2010

The $5 Guerrilla User Test

by Hang

The internet is all atwitter about ReadWriteWeb’s article on how a quirk in Google had them highly ranked for the search term “Facebook login”.

It was like we had unearthed a long-lost city, the Atlantis of the Internet. But instead of treasures and gold we’d found a steady deluge of confused and frustrated users who had tried everything they knew to do and just wanted to log in to Facebook, damnit. But how had this happened? It certainly wasn’t that thousands and thousands of people had just started searching for “facebook login” yesterday. This stream of people has been there all along and something is broken.

There is a persistent meme, which this article is only helping reinforce, that user experience professionals are needed because the average user is far less intelligent than the average designer so we need to hire some people who can “think stupid” just like the user. Such thinking doesn’t benefit anyone, developers lose respect for the user and start creating condescending, dumbed down UIs. Users continue to find the new software hard to use because it doesn’t address their core issues.

Users aren’t so much unintelligent as they are distracted and indifferent

Your average user may be perfectly competent and zip through your app like a charm when they’re in a controlled setting, focusing exclusively on your application and incentivized to succeed. But such a scenario is almost never likely to happen in the real world. What’s more realistic is that they’re devoting, at best, 10% of their attention towards your app while they have the TV blaring in the background, an IM conversation they’re also involved in, thoughts about whether that meeting with the boss tomorrow means a promotion or getting fired. Your application is at best, 5th on their priority list and they’re largely moving on autopilot as they navigate through it. Once you understand this basic reality, user behavior becomes a lot easier to understand.

A couple of years ago, a grizzled UX professional taught me one invaluable fact.

Drunk people are a pretty accurate mimic of distracted, indifferent people

This insight has lead to a wonderful technique I’ve been refining over the years that I call “The $5 Guerrilla User Test”.

The $5 Guerrilla User Test

The $5 Guerrilla User Test

Here’s the 5 second version:

  1. Bring a laptop to a bar
  2. Offer to buy someone a beer in exchange for participating in a user study
  3. Watch your application crash & burn as people do all sorts of ridiculous ass shit they would never do in a lab but constantly do in real life
  4. Go back, apply the lessons you have learnt, repeat until you have an app that is 100% drunk person proof

This is the slightly longer version for those who are interested:

  • Like conventional user studies this is best done with a group of two, one to run through the script, the other to take notes.
  • Approach in a friendly manner, explain who you are and who you work for and ask them if you can have a moment of their time.
  • If they don’t seem receptive from the get go, thank them for their time and move on to a different target
  • Explain to them that you want some insight on a piece of software you’re currently building and tell them that you’re willing to buy them a pint of beer as compensation for their participation
  • If they accept (and 90% of them will), ask them their preference of beer and then ask your partner to go off and order it
  • While you’re waiting for your partner, inform them of your data collection policies, the procedure and the standard stuff about how they can quit at any time. It doesn’t much matter what exactly you say to them, the key is to make it boring. This step is key. When you first approached them, you were something novel for the night so they’re interested and motivated to perform. 2 minutes of dull chatter is going to lose their attention and they’re back to being utterly indifferent about your problems again.
  • Once your partner gets back, run it just like any other user study.
  • At the end, hand them their beer, thank them for their time and drop off some business cards for some easy word of mouth marketing. If you offer any sort of premium features, give them a year’s access to it as a gift as well. It’s a nice surprise and converts surprisingly well.

That’s it! It’s cheap, fast, can be done by anyone and gives you insights you never would have gotten hiring a professional usability consulting firm. Go out and do it!

Here’s a couple more tips I’ve picked up over the years:

  • The first time you do it, you’re probably going to be suffering from approach anxiety. Start off approaching a group of the same sex as you so that the encounter isn’t sexualized. Next, move on to a mixed sex group and then finish the night with an opposite gender interaction so you get a nice demographic spread.
  • Focus on people who don’t normally get talked to at bars, middle aged people, homely girls, the guy sitting in the corner.
  • People alone, reading a book, have a 50% chance of agreeing to participate. People alone, reading a newspaper, have a 99% chance of agreeing to participate.
  • If you ever get the feeling that you’re being messed with, politely end the experiment, give them their beer and move on to the next round of testing.
  • Groups of all guys tend to be the only ones who ever mess with you. I avoid asking them as a rule.
  • You want someone who’s pretty drunk but not completely trashed. A good way to calibrate is to keep on asking progressively drunker people until your results become garbage, then back off one notch from there. After a few rounds, you learn to spot the signs of an ideal participant.
  • Conveniently for you, weekdays end up being more effective than weekends so you can do this after work.
  • I generally try to keep the entire session down to around an hour which works out to be roughly 4 * 15 minute user studies. Much more than that is tiring and too much data to analyze the next day.
  • While beer is good before & after the user study, try and keep beer away during the user study. Spilled beer on a laptop can be an expensive mistake.
  • Bars are wonderful at segmenting by demographic. Match the bar you’re going to with the user population you want to target. Different bars will produce slightly different results but the variation is not huge.
  • If you’re planning on regularly using one bar to do your tests, tip the bartender well from the start, like, 30 – 40%. That extra $1 you spend isn’t going to break the bank and having a bartender on your side brings all sorts of benefits.
  • edit: One extra tip from a friend, mobile app development benefits even more from this technique since mobile use tends to involve even more distraction. Users are using your app while walking down the street, driving, holding a conversation etc.

Anyway, that’s it. I’d love to hear from people how their experience with the $5 Guerrilla User Test goes.

Google’s lead visual designer quit due to a clash of cultures

by Hang

Douglas Bowman, Google’s lead visual designer announced yesterday that he was leaving Google to join Twitter. At the root of it, Bowman’s decision to leave stems from a clash of cultures between the world of Interaction and Visual Design. The best way to understand this this clash of cultures is to listen to the ghost stories each field tells the young’uns.

In Interaction Design, around the campfires at night, it’s common to hear a variant of this chilling tale:

I heard, there was this company once, where they, like, got these totally great designers to build this user interface for them and they were all excited about it being the best thing since sliced toast until they tried to watch some people use it in the real world and it, like, totally sucked. The things everyone thought were easy to use were completely confusing. Luckily, they went through several iterations of redesign and testing the thing until it became something users loved.

Interaction designers are actively trained to filter out expert opinion as a justification for design decisions. The expert, no matter how qualified and trained they are, is ultimately, not the user and is ultimately, totally ineffectual and predicting what the user is like. The only way that design decisions can be justified is through feedback from actual users. Uttering the words “I prefer…” as justification for a design decision is the quickest way to move you from the potentially-an-ally category to dangerous-fool-who-must-be-neutralized category in the eyes of an interaction designer.

Over in the Visual Designer camp, a different ghost story is being passed round the campfire:

I heard, there was this company once who hired this, like, genius visual designer who built them this totally bold and brilliant design. But then, in an attempt to please everyone, the design was buried under so many focus groups and QA evaluations that  integrity of the design was destroyed and what was ultimately put up, like, totally sucked and ended up pleasing no one. Luckily, a more design friendly management was put into place and the original design was restored which ended up creating the emotional bond with the users that saved the company.

Visual designers are trained to keep their artistic integrity in the face of pressure and to be the keepers of the secret knowledge against the tide of the aesthetically ignorant. Uttering the words “consensus seeking” as justification for a design decision is the quickest way for you to become a dangerous-fool-who-must-be-neutralized in the eyes of a visual designer.

You can see both of these dynamics play out in the Google saga. Douglas Bowman’s characterization of the design process at Google:

Yes, it’s true that a team at Google couldn’t decide between two blues, so they’re testing 41 shades between each blue to see which one performs better. I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that. I’ve grown tired of debating such miniscule design decisions. There are more exciting design problems in this world to tackle.

The debate on border pixels dragged on because Bowman became a dangerous-fool-who-must-be-neutralized in the eyes of the interaction design team.

Similarly, on Marissa Mayer’s attempt to reach out towards the visual designers:

A designer, Jamie Divine, had picked out a blue that everyone on his team liked. But a product manager tested a different color with users and found they were more likely to click on the toolbar if it was painted a greener shade.

As trivial as color choices might seem, clicks are a key part of Google’s revenue stream, and anything that enhances clicks means more money. Mr. Divine’s team resisted the greener hue, so Ms. Mayer split the difference by choosing a shade halfway between those of the two camps.

Is so, tin-earred it’s cringe inducing. Like rich yuppies trying to connect with the less affluent by speaking the language of the “street”, Marissa reads the culture of visual design so wrong and her attempt and consensus and compromise ends up doing more harm than good.

The sad thing is, both of these viewpoints are perfectly justified and are the result of a counter-intuitive lesson learned. Both of these ghost stories are repeated precisely so the newbies in the field don’t end up making the same mistakes the pros once made. Unfortunately this means for both sides, the views of the other side look like ignorance.

Look, I was like you once, and then I learned better. So I’m just going to sit hear and wait for the other shoe to drop for you Mmmkay? Do you want to hear a ghost story while we’re waiting?

So what you end up getting is a staring contest where each side is waiting for the other to finally blink. Unfortunately, in this case, Douglas Bowman blinked first and both Douglas and Google were both impoverished for this.

PS: In anticipation of the criticism that I have no business talking about visual design when the design of my own site sucks so much, I know, it’s being fixed, be patient.

Nov 8th (day 26): The state of Academic HCI

by Hang

Jeff Atwood’s blog post on reading HCI Remixed lead me to try and clarify some of the thoughts I’ve been having on the role of Academic HCI and it’s relationship with developers, entrepreneurs and other interested parties in this space. I’m an enormous fan of the book and I know and admire many of people who have contributed essays to it but it’s never struck me as a book that would be of much use to those outside of the tight knit community of academic HCI researchers. On reflection, I’ve noticed an interesting distinction which might not be immediately apparent to outside observers.

The normal role of (good) academic research is to engage in medium to long term basic research which will eventually migrate it’s way into industrial research and finally into products. Academic material scientists are working on carbon nanotubes which will eventually be thrown over the wall to practising material scientists to make into space elevators. Academic biotechnologists are working on sequencing genomes to throw over the wall to practising biotechnologists to convert into gene therapy. Natural, the naive observer might expect that the role of Academic HCI is to develop new tools and techniques that practising HCI professionals can then take forward and use.

In actuality, the worlds of Academic HCI (including “Industrial Research”) and Professional HCI have very little to do with each other. Academic HCI is the province of major academic universities as well as industry research labs such as Microsoft Research, IBM and Xerox Parc. Professional HCI is largely the province of Interaction Designers, User Experience Engineers and Usability Experts who work for either large companies of consultancy firms.

The key to understanding Academic HCI is that it’s not in the business of throwing stuff over the wall to HCI folk, it’s main goal is to throw research over to product designers. Academic HCI is in the business of envisioning potential future products that have some significant interface component. This is a key distinction to make and one which I failed to adequately understand when I first entered my PhD program, focusing on HCI.

Indeed, there really is no discipline dedicated to advancing the state of the art of practicing HCI and I suspect a large part of this is because the slot of “Academic HCI” has already been taken. The work of contributing to a greater theoretical and practical understanding of the new problems facing design is one which simply isn’t being done for lack of various infrastructure elements like funding, tenure and journals.

Although Academic HCI and professional HCI share the same names and even aspects of common terminology, it’s a mistake to see one as the research version of the other. I found out the hard way that the field I was looking to make a contribution in simply doesn’t exist and that was primarily the reason I decided to leave academic and strike out on my own as an entrepreneur.

August 15 2008

HCI and blogging

by Hang

How do we apply HCI and UCD processes to blogging?

Faceted blogging was a HCI inspired idea, what else?

Would personas help?

Copyright ©2009 BumblebeeLabs — Theme designed by Michael Amini