A Definitive Look at Collaborative Online Learning

Book A Demo
Resources / Blog

Sample Usability Test Plan

This sample provides a blueprint for conducting a site design usability test.

Usability Test Plan

[site name here]

Topic            

Description

What is Usability Testing?

Usability is about how easy a website is to use. The father of usability, Jakob Nielsen, explains it like this. “On the Web, usability is a necessary condition for survival. If a website is difficult to use, people leave. If the homepage fails to clearly state what [you offer] and what users can do on the site, people leave. If users get lost on a website, they leave. If a website's information is hard to read or doesn't answer users' key questions, they leave.”  Notice a pattern?  A site has to be useful, and pleasing, for people to stay.

Usability testing isn’t just asking for feedback, which is often hypothetical and unreliable.  It’s employing the think aloud protocol to observe users while they attempt to perform key tasks on your site.

Want to know more?

Participant selection

We need 4-5 participants per course who are representative end users and meet the following criteria:

  • come up with your own criteria…examples below
  • New to role
    • Mix of ____ (could be generation, experience, familiarity with content, country of origin, other demographics)
  • Current in role
    • Mix of ____

Logistics

  • Each participant should be on a device they would use to access the site (e.g., laptop, tablet, phone) and in an environment in which they would normally access the site (e.g., at their desk).
  • Each session will:
    • Require 45-60 minutes
    • Be conducted virtually via web conference and recorded (recordings will be available to the full project team)
    • Be led by [names]
  • Optionally
    • Include another team member to take notes and ask follow-up questions as needed
  • Include observer(s) from [client]
  • Follow the same test script (below) and use the same site (i.e., no changes to the site will be made during usability testing)
  • Testing will be conducted during [date range].
  • After testing has been completed, feedback will be aggregated for review by the project team. The team will decide what changes should be made, and those changes will be made along with final feedback from the project team and clients.

Research questions

  • Site overview
    • How do users respond to the elevator pitch (i.e., description of site purpose and function)?
    • How well does the site meet users’ expectations, as created by the elevator pitch and any other assumptions they have?
    • How likely are users to engage with the site
  • Organization and titling
    • How clear and logical are panel titles and categories to users? What expectations do those labels create?
    • What are users’ reactions to the titles and descriptions on tiles (e.g., individual tiles and learning paths)? How well do those labels meet expectations for what learners need and want?
    • How do they react to the organization of content?
    • How easily are users able to get to what they need on the site?
  • Content
    • What content were users most interested in?
    • How did users respond to missions and assignments?

 

 

Email Invitation to Participants

(personalize as appropriate)

Subject line: $75 Amazon card for 60-min study – are you interested?

Hi [name],

I’m working on a project to build online learning experiences to help [elevator pitch on site’s purpose here]

We’re seeking people to give us feedback on what we’ve developed so far.  The team asked me to recommend people who are really articulate, and I thought of you.

The study works like this

  • We schedule a 60-minute webmeeting during which we’ll ask you to share your computer screen, log into the site, and give us your honest, immediate reactions
  • No prep is necessary – just make sure you’re in a quiet place
  • We send you a $75 Amazon gift card as a token of our thanks

 

These are the time slots we have available.  If you think you could participate, let me know which time works for you and I’ll see if it’s still open.  If it is, I’ll connect you to [name] from Intrepid Learning who will send you meeting details. (Please add [email address] to your safe senders list)

(A shared doc will be created for everyone who will send invitations)

Date

Time

Time

 

Name

Name

For Example,

Mon, Feb 1

10:00 - 11:00

11:30 - 12:30

 

Eric Brown

Neha Sehgal

 

Test Script

The following is written as a script for illustration purposes. However, the Intrepid consultant will deliver it in a conversational style and adapt it as needed.

  1. Welcome and introductions

Thanks for joining the call. I’m <name>, and I’ll be leading today’s usability session. My colleague <name> is also with us to help take notes and ask a few questions. <Do brief introductions: name, employer, location, brief description of role.>

  1. Introduction to the testing procedures

We’d like to ask you permission to record your screen activity and verbal feedback during the session, so we can make sure we’ve accurately captured your feedback. Is that alright?

Let’s start by re-capping the purpose of this session. We’re interested in your honest, in-the-moment feedback to an online learning experience called <program name>. The purpose of the site is <describe in a few words>. We are in the early stages of the project, and your feedback will help us identify improvements to site layout.

In this call, I’m going to ask you to log into a site; I’ll put the URL into the web meeting chat window in just a minute. I’ll ask you questions along the way and would appreciate your candid feedback. I’ll ask you to wait for my cue before clicking on anything in the site.

Most importantly, I’m going to ask you to think out loud, voicing the things that we often say to ourselves when we see a new website, such as:

  • “Oh, I think I’ll click here.”
  • “That’s not what I expected to happen.”
  • “I don’t understand what this is supposed to mean.”
  • “I’d use this, or I wouldn’t use this because…”

It’s our job to figure out what you need in this site, not your job to figure out how we designed the site. That’s why your “thinking out loud” reactions are so important. You can’t hurt our feelings; we need you to be completely honest with your feedback. If anything seems confusing, not useful, etc., please let us know!

  1. Your role in the testing process

While we expect this to be fun, please don’t hesitate to let us know if you feel uncomfortable. You can skip any of the questions we ask or quit the session at any time. And there are no wrong answers! No pressure, we’re just interested in your honest reactions to what you see.

We are testing the website to find out what works and doesn’t work. We’ll be going through this same testing process with a number of folks like you and comparing the results. Our focus will be on trends that develop.

If something seems confusing, it probably indicates a problem with the design. As a result, we want you to point those things out because that’s exactly what we are trying to identify and improve.

When this happens, I won’t give you any guidance on what to click, because we're interested in listening to you describe what you are experiencing and what you’d like to do. At the end of the session, I'll ask you a few summary questions, and then we can talk about any remaining issues or concerns you have.

  1. Test scenarios

Please click “share desktop” so we can watch your screen activity. We’ve posted the URL and a username and password in the web meeting chat window. Please click the URL to open the site. Then enter the username and password to login.

Home screen

[Capture what the participant is interested in and probe on what drives them toward it.]

What is your eye drawn to initially on this page?

Based on the options available to you on this page, what interests you the most? What would you be likely to explore?

What would you like to do next on this page?

What would you expect to find under each of the six panels:

  • Panel 1
  • Panel 2
  • Panel 3
  • Panel 4
  • Panel 5
  • Panel 6

Panel 1 <select those questions relevant for the panel>

“I’ll give you a few moments to browse the page. 

What is your eye drawn to on the page?

Is it clear what’s clickable?

Are the terms useful?  (if they don’t mention the buttons, ask if they noticed them)

When you click the ______, do you get what you expected?

  • buttons/accordions
  • learning path
  • content tiles (try to check all content types at some point during the study)

When you click into the learning path, do you get what you expected?

Which of the titles look most interesting to you?

When you click the tiles, do you get what you expected?

Would you find these resources/activities useful?

<Repeat for each panel>

  1. Open-ended Feedback

Do you have any issues or concerns we have not discussed?

What types of additional content would you like to see in this site?

  1. Thank you and close

Thank you for making the time to talk to us. Your feedback has been extremely helpful. We really appreciate your contribution to this effort.

false LOREM IPSUM TITLE

RELATED RESOURCES

EXECUTIVE SUMMARY

EXECUTIVE SUMMARY

Resources for Migrating Your ILT Online

WEBINAR

WEBINAR

L&D Training Myths, Busted!