Archive for the ‘usability testing’ Tag

Prototyping with your CMS

We use the open-source content management system Plone to build our site, and have been very pleased with how powerful and user-friendly it is. In the process of redesigning our site, we’ve conducted wireframe tests and settled on our visual design and information architecture. Now we want to build a more advanced prototype, to test how this design will really look and feel in a browser.

To do this, we set up a copy of our Plone site on another server so we’d have one that we could freely modify and change for testing purposes. The final result ends up feeling very much like we think the redesigned site will feel, but we still have a ton of flexibility to move things around, change what doesn’t work, and correct problems where our users get stuck.

Does anyone else use their CMS as an advanced prototyping tool?

Scenario Creation

So now I’ve done some interviews with current users, I’ve got my survey and card sort results, I’ve crafted personas of the major user constituencies, and I’ve shown some wireframes to potential users to make sure the navigation fits their mental model.

What next? Taking a page from Alan Cooper’s “Goal-Directed Design” method I’ve created some high level scenarios that convey how the new site could ideally interact with our users and help them accomplish their goals. Mr. Cooper calls these scenarios “context scenarios” and one of mine looks like this:

  1. Don gets an assignment to perform a training session for parents on identifying gang-related behavior at a nearby school.
  2. Don visits and looks up information about gangs. Since he has been to before, the site remembers that he’s a law enforcement officer and displays content appropriate for him.
  3. In the Gangs section, Don sees emerging trends about gangs, training materials about gangs, NCPC products relevant to gang prevention, and NCPC programs that help prevent gang-related crime.
  4. Don skims the emerging trends and then looks through the training materials. He finds a training session geared toward parents and downloads the PowerPoint slides.
  5. After conducting his training, Don shares his experience and comments on the material at

This scenario is purely textual, it mentions nothing about specific interaction details on the site. It’s used to describe what an ideal interaction might feel like to our user.

Then, I took my context scenarios and matched them up with the relevent wireframes to create task scenarios. Task scenarios are much more detailed flows of how the user would typically progress through the interface to accomplish their tasks (we think). Once the steps and screens are put together, I’ll take my basic prototype and see if a user can accomplish the steps in the scenario, and if they do it in the way I expect them to.

The feedback I get from those tasks will lead to design changes I’ll put into the next, more advanced set of prototypes.

Are you finding the right participants?

Many UX professionals seem to differ when it comes to finding the right participants for usability testing. Can you just grab anyone? Or do the subjects need to be existing users?

Harry Brignul illustrates the dangers of using just anybody as a usability test participant on his blog, with some convincing examples.

On the other hand, Matt Brown from says he’s gotten some good feedback from his admittedly “loose, informal user testing” at a coffee shop mixing existing users with random passersby.

Who’s right? Obviously, it depends.

Harry’s example uses a specialized application for a certain group of sales people. Matt is testing designs for a music site aimed at the general public.

If you’re targeting the general public, you want to make sure you get a good mix of demographics in your user base. If your application is for a specific audience, not testing with members of that audience is an insanely risky proposition.

Most likely you’ll fall somewhere in between. At NCPC, we serve a lot of law enforcement personnel and folks interested in community improvement, but we also have materials for parents, teenagers, and kids. We need to make sure our offering is flexible enough to handle our disparate audiences, so we have to try and test with all of them.