Usability research 

Getting feedback on our products and services from the people we serve

The Centers for Medicare & Medicaid Services (CMS) interacts with millions of people every day.  The Digital Service at CMS (DSAC) conducts usability research to learn how easy or difficult these touch points are for the people who rely on them.

What usability testing is and how it works 

When our team is looking for feedback on a process, service, or tool, we reach out to members of the public to give us feedback. Usability testing is way for us to evaluate how easy something is to use. 

Typically we ask an individual to complete a few tasks on a website, app, or prototype. We observe how hard it is for them to complete the tasks, which tells us what design changes are needed. We also ask follow up questions to understand what's working well or not from the individual's perspective. 

After doing this with this with anywhere from 3 - 10 individuals, we usually have a clear sense of how to improve the design. Here are some answers to common questions we receive about usability testing: 

Who will be there? 

There will be up to 4 CMS employees on the call and the research participant (the person we’re receiving feedback from).  A typical research session will look like:

• CMS facilitator
  • This person will ask most of the questions and lead research participants through the tasks.
• CMS note-taker 
  • This person will take most of the notes.
• CMS observers (optional)
  • These are people who are trying to learn by watching how well the website, app, prototype, etc. is working. 
• Reserach participant
  • The person we’re receiving feedback from.

What will the session be like?

Each session will be a little different to meet the needs of our audiences and the goal of the session but will include these parts:

Introduction

The CMS facilitator will introduce the team, ask the participant for consent to record and take notes, and ask a couple of general questions of the participant. 

Tasks and feedback about the tool

The CMS facilitator will then share the product or information about the service with the participant to provide feedback on. The facilitator will ask the participant to complete some tasks. Follow up questions are generally open ended to better understand the participant's experience. You might hear questions such as:

• “What do you think you could do with this?”
• “Could you tell me more?”
• “Could you share what was frustrating?”
• “Was there anything that was confusing?”

Wrap up and last thoughts

This is an opportunity for the participant to share feedback that didn’t come up in the session. Every person will have experiences to share, which is important to making these touch points better. The CMS note-taker and observer will also have the opportunity to ask any follow-up questions.

After the final questions, the CMS team will check in one more time to ensure the participant is comfortable with us sharing the recording with teammates, ask it's ok to use anonymous quote, and offer any support resources.

Remote usability feedback sessions

Remote sessions allow CMS to receive feedback from people across the country. When organizing these sessions the CMS facilitator will often conduct outreach and provide resources ahead of time. These resources include:

• Session information. Which includes:
  • What topic we’re asking for feedback on
  • When the usability session will occur
  • How long the usability session will take
• Contact information:
  • Life happens. We’ll share our contact information just in case a participant needs to get a hold of us to reschedule.
• Any consent or other documentation:
  • Sometimes we require consent forms to ensure that participants are intentionally opting into feedback sessions.

We know that sometimes the topics we’re talking about can be hard. At any point during these sessions participants can pause or stop entirely, for any reason.