Prototyping with your CMS

We use the open-source content management system Plone to build our site, and have been very pleased with how powerful and user-friendly it is. In the process of redesigning our site, we’ve conducted wireframe tests and settled on our visual design and information architecture. Now we want to build a more advanced prototype, to test how this design will really look and feel in a browser.

To do this, we set up a copy of our Plone site on another server so we’d have one that we could freely modify and change for testing purposes. The final result ends up feeling very much like we think the redesigned site will feel, but we still have a ton of flexibility to move things around, change what doesn’t work, and correct problems where our users get stuck.

Does anyone else use their CMS as an advanced prototyping tool?

Recognizing Deeper User Goals

In the process of redesigning our site, we’re putting a lot of effort into connecting with our users’ emotions. Our users are people trying to prevent crime in their schools, communities, and workplaces. We should do more than present information to them, we also need to put that information in context and convey the enormous importance of the work that our users are doing to keep people safe with the resources we provide.

This reaches to the core of our users’ goals. In About Face 3 Alan Cooper gets to the heart of this:

Users’ goals are often quite different from what we might guess them to be. For example, we might think that an accounting clerk’s goal is to process invoices efficiently. This is probably not true. Efficient invoice processing is more likely the goal of the clerk’s employer. The clerk is more likely concentrating on goals like appearing competent at his job and keeping himself engaged with his work while performing routine and repetitive tasks, although he may not verbally (or even consciously) acknowledge this.

In interviews, I’ve spoken with crime prevention practitioners who are using the materials we provide on our site to educate kids and work with law enforcement to help prevent crime. We give them the tools that they need to do their jobs, but there’s also a huge opportunity there to recognize them for keeping people across the country safe, and turning at-risk individuals away from criminal activity. By expressing recognition and gratitude to the people who perform this important work, we can help fulfill their deeper goals – to be appreciated for helping their communities and to be reassured that they’re making a difference (which they are). In turn, they will feel much more engaged by the site.

There are several ways we can attempt to do this:

  1. Highlight images and stories of our crime prevention programs in action.
  2. Emphasize the real impact that crime prevention efforts are having with statistics and examples.
  3. Promote the work that crime prevention practitioners are doing with our help, rather than just promoting what we’re doing.

In my next post I’ll write about how we will meet our users’ goals of communicating, collaborating, and sharing with other practitioners in the field with more interactive and user-generated content combined with some social networking integration.

The Importance of Documentation

The title of this post comes off a lot like “The Importance of Eating Your Vegetables”, but I think when you work for a non-profit this really is important.

During the process of researching our users, listening to our stakeholders, developing scenarios and personas, drawing wireframes, and all the other good stuff that goes into developing our website we’ve made a ton of decisions. We’ve weighed different alternatives, had great insights, and ditched some ideas we used to think were great insights. The ramifications of those decisions are obvious to us, but what about the people who follow us?

If we’re replaced, we want new people to at least understand where we were coming from even if they decide to go in a totally different direction. Thus, it’s important that we properly document all of our UX activities. I’m not talking about writing a thick binder that will sit on a shelf somewhere and never get read.
Our web team uses an Assembla wiki to track our activities, describe our research results, and explain the rationale behind our UX decisions. Anyone who comes along later and wonders what questions we asked during our user interviews, or why we made certain changes to our site navigation will be able to get those answers.
That helps the organization by reducing the impetus for new web staff to re-invent the wheel because they don’t understand the history of our site development. The less re-inventing our organization does (without a good reason) the more efficient it is at accomplishing its goals in the real world.

Scenario Creation

So now I’ve done some interviews with current users, I’ve got my survey and card sort results, I’ve crafted personas of the major user constituencies, and I’ve shown some wireframes to potential users to make sure the navigation fits their mental model.

What next? Taking a page from Alan Cooper’s “Goal-Directed Design” method I’ve created some high level scenarios that convey how the new site could ideally interact with our users and help them accomplish their goals. Mr. Cooper calls these scenarios “context scenarios” and one of mine looks like this:

  1. Don gets an assignment to perform a training session for parents on identifying gang-related behavior at a nearby school.
  2. Don visits and looks up information about gangs. Since he has been to before, the site remembers that he’s a law enforcement officer and displays content appropriate for him.
  3. In the Gangs section, Don sees emerging trends about gangs, training materials about gangs, NCPC products relevant to gang prevention, and NCPC programs that help prevent gang-related crime.
  4. Don skims the emerging trends and then looks through the training materials. He finds a training session geared toward parents and downloads the PowerPoint slides.
  5. After conducting his training, Don shares his experience and comments on the material at

This scenario is purely textual, it mentions nothing about specific interaction details on the site. It’s used to describe what an ideal interaction might feel like to our user.

Then, I took my context scenarios and matched them up with the relevent wireframes to create task scenarios. Task scenarios are much more detailed flows of how the user would typically progress through the interface to accomplish their tasks (we think). Once the steps and screens are put together, I’ll take my basic prototype and see if a user can accomplish the steps in the scenario, and if they do it in the way I expect them to.

The feedback I get from those tasks will lead to design changes I’ll put into the next, more advanced set of prototypes.

Are you finding the right participants?

Many UX professionals seem to differ when it comes to finding the right participants for usability testing. Can you just grab anyone? Or do the subjects need to be existing users?

Harry Brignul illustrates the dangers of using just anybody as a usability test participant on his blog, with some convincing examples.

On the other hand, Matt Brown from says he’s gotten some good feedback from his admittedly “loose, informal user testing” at a coffee shop mixing existing users with random passersby.

Who’s right? Obviously, it depends.

Harry’s example uses a specialized application for a certain group of sales people. Matt is testing designs for a music site aimed at the general public.

If you’re targeting the general public, you want to make sure you get a good mix of demographics in your user base. If your application is for a specific audience, not testing with members of that audience is an insanely risky proposition.

Most likely you’ll fall somewhere in between. At NCPC, we serve a lot of law enforcement personnel and folks interested in community improvement, but we also have materials for parents, teenagers, and kids. We need to make sure our offering is flexible enough to handle our disparate audiences, so we have to try and test with all of them.

Ad Council Partner Conference – Social Media

Today I got the opportunity to go to the Ad Council Partner Conference at the National Association of Broadcasters. The presenters in order of appearance were Ian Schafer (Deep Focus), Michael Becker (iLoop Mobile), John Roland & Tim Conley (Extreme Reach), and Maurice Boissiere (Clear Spring).

Overall the presenters did a very good job. Mr. Schafer said he’d post his slides on his website later today.

It was also pretty cool that one of our PSAs was used by John Roland as an example of a viral video calling attention to a worthwhile cause.

One neat thing that Michael Becker brought up was the ability of 501(c)3 organizations to accept donations via the next generation of sms text messages, though I wish it had been part of his main presentation (which was still the most intriguing in my view).

There were a lot of mentions of Facebook, MySpace, and YouTube though I would’ve liked to see a few more examples of what organizations can do with their own web properties in terms of creating a unified user experience that ties into social media. There were some good examples, especially in relation to mobile devices but there were also a lot of generalities.

Getting Interview Subjects to Talk More

What do you do when you’re interviewing someone, and everything is going swimmingly until you realize that the same answers keep coming out and they’re about 5 words each?

Some people just like to talk more than others – I know I tend to be on the quiet side myself. I’d probably be a terrible interview subject.

So how could someone draw me out without annoying me by asking the same questions over and over?

You’ll need to establish a level of comfort with the subject. This can be very difficult when you’re speaking to them over the phone, especially with all the obligatory things you’ll need to say like “Is it ok if I record this conversation?”

When you first call them up greet them, thank them, and bring up some sort of icebreaker topic. Once you think they’re speaking fluidly you can get started with the real questions.

If they give you a short, blase answer, try asking “Could you tell me a little more about that?”

Elaborate on what they say, paraphrase their answers back to them, and see if that sparks more comments from them.

What suggestions do you have for this problem?

Contextual Interviews vs Regular Interviews

Many user experience practitioners sing the praises of contextual interviews, and with good reason. By traveling to a place where a real user sits and employs your system, and watching them interact with it, you gain insight into their process that you would miss by any other method.

Contextual interviewing can also be expensive with the travel expense of visiting the willing participant and the cost of whatever recording equipment you want to use.

Can the same results be gained by interviewing people over the phone while they use the system? Probably not, but you might come close.

We’re reaching out to some of the users who participated in our surveys and indicated that they’d like to be contacted again to conduct some interviews with us. They’re spread across the country, which we like because our audience is nationwide.

We considered using a remote screen capture tool like TechSmith’s UserVue, and we still might if we feel that we’re missing out on too much data. But for now, we’re going to speak with our participants on the phone while they use our site, and follow along as best we can on our end. Our conference room equipment allows us to record phone conversations, so we’ll be able to keep our interviews for later use.

Note: Always get permission before recording a phone conversation.

Silverback is pretty cool for the price

For guerrilla usability testing (the only kind of usability testing most non-profits can afford)Clearleft’s Silverback seems like a pretty good solution. We tested it out today in preparation for some prototype testing, and once we had our webcam hooked up it worked very smoothly. The interface is simple and straightforward, and at $50 the cost can’t be beat.

On the downside, you seem to have to export your recordings to mov files before you can play them back. This can take several minutes per recording. It would also be nice if there was a way to organize all your recordings within the App more effectively. Overall, it’s a simple app that accomplishes its goals admirably.

Creating Information Hierarchies

Now that we’ve done some research, patterns of user behavior are beginning to emerge. A majority of our users prefer that information be grouped by audience (teens, law enforcement, seniors, etc) while a significant portion prefer that they be grouped by topic (gangs, fraud, drug abuse, etc).

We originally predicted that this would happen and believed that we would need to provide alternate navigation options so that people could find resources by either topic or audience.

However, in our case the more I think about it the more I think audience should be like a filter that applies to the topic based navigation scheme. For instance, teens and law enforcement personnel might both be interested in information on drug abuse, but their perspective needs to be taken into account to provide the appropriate materials for each.

It will be interesting to see how we can turn this insight into a working prototype. I’ll post it here when it’s ready!