Are you finding the right participants?

Many UX professionals seem to differ when it comes to finding the right participants for usability testing. Can you just grab anyone? Or do the subjects need to be existing users?

Harry Brignul illustrates the dangers of using just anybody as a usability test participant on his blog, with some convincing examples.

On the other hand, Matt Brown from says he’s gotten some good feedback from his admittedly “loose, informal user testing” at a coffee shop mixing existing users with random passersby.

Who’s right? Obviously, it depends.

Harry’s example uses a specialized application for a certain group of sales people. Matt is testing designs for a music site aimed at the general public.

If you’re targeting the general public, you want to make sure you get a good mix of demographics in your user base. If your application is for a specific audience, not testing with members of that audience is an insanely risky proposition.

Most likely you’ll fall somewhere in between. At NCPC, we serve a lot of law enforcement personnel and folks interested in community improvement, but we also have materials for parents, teenagers, and kids. We need to make sure our offering is flexible enough to handle our disparate audiences, so we have to try and test with all of them.

2 comments so far

  1. Harry on

    Hi Jason,

    Thanks for the mention. That post I wrote seemed to stir up a bit of controversy, interestingly enough.

    I totally agree with your post – most clients are likely to be in the middle. The client I wrote about should have found it easy to succeed with such a tightly defined user-base, but they forgot to speak to them during the design process.

    Taking a quick look at Matt Brown’s post on lastFM, he says he recruited “Long-time users who’ve been with us for years; new users still discovering what we do; friends and relatives; random people […]”

    Two of the four in that list are very tightly screened recruits. I’d bet my bottom dollar that he wouldn’t make any substantial design changes following feedback from randoms who don’t even have a personal music player.

    LastFM didn’t grow it’s mass appeal by starting out aiming for the masses. The masses took a while to catch up.

  2. jasonwilkens on

    That’s a great point, thanks for the comment!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: