User Research, Prototyping, Usability Testing

Spring 2015

“How might we design a feedback system that would encourage radiologists to continue learning and improving their performance throughout their careers?

For my HCI Capstone I worked with Alex Stern, Salem Hilal, Aya Demler, and Sue Park. Our client was a Professor of Biomedical Informatics at the University of Pittsburgh. As Team Lead, I was especially responsible for leading concept evaluation and usability testing, ensuring we were moving along to meet deadlines, and doing a full pass on editing reports and presentations.

Final Video - Proof of Concept

password: radpath

1. Exploratory Research

For our initial phase of research we conducted interviews with radiologists, a literature review, and synthesized our findings in an affinity diagram and customer journey map. 

The Interviews

The interviews were critical in providing us insights into the types of tasks accomplished during a radiologist’s workday, the frequency of daily tasks and approaches, as well as the process of bookmarking and saving cases. It also allowed our interviewees to educate us on factors we did not initially consider, such as the differences between private practice and teaching hospital environments. 

Literature Review

We examined literary texts in radiology of best practices in radiology, feedback in learning systems, and informatics in healthcare to build context for the project. Through literature reviews, we were able to better understand what types of feedback systems would work, and why previous ones may have failed. Additionally, it provided us with a framework to help drive the development of our initial prototypes. 

Affinity Diagram

We constructed an affinity diagram to synthesize and surface findings from our research. By organizing all of our research notes through the affinity diagram, we were able to gather six key insights, discussed more in depth below. 

Content Journey & Flow Diagram

Based on our interviews and literature reviews of current radiological practices, we created a content journey map that depicts a step-by-step account of a patient’s medical record passing through various stakeholders in the medical process. This customer sequence map and the flow digram showed us the role of the radiologist in relation to the patient’s care. It also shed light on the fact that radiologist have a lack of a structured feedback loop in understanding the final outcomes of the patient’s health. 

Insights From Research

Based off the initial phase of research we discovered 6 key insights about radiologists in terms of feedback. These insights provide the guidelines and requirements our solution must meet in order to be an effective tool in encouraging radiologists to learn and improve. 

  1. Disjoint sources of information cause problems
  2. Feedback is useful to radiologists 
  3. Radiologists need feedback appropriately
  4. Radiologists are working at capacity
  5. Radiologists are open to and seek feedback
  6. Radiologists are motivated to perform well 

2. Low-fidelity prototyping

We began this next phase by developing a set of pain points we would push radiologists on in order to understand the boundaries of our solution. Our pain points investigated: 

  1. Being evaluated against peers 
  2. The timing of notifications 
  3. The effectiveness of feedback away from work 
  4. Individual vs. aggregated notifications 
  5. The value of immediate alerts on discordant cases 
  6. The usefulness of referencing prior cases 
  7. The benefits of seeing a compiled list of bookmarked cases 

Storyboarding & Speed Dating

The goal of the first round of storyboarding was to test the above pain points. We speed-dated the ideas with 5 radiologists in a radiology reading room, after spending the day doing contextual inquiry. The initial storyboards led us to form a final set of storyboards and a service blueprint that we ran by two engineers at UPMC's Technology Development Center and our client as a proof of concept. The positive feedback enabled us move forward in the prototype development process. 

1st Round Speed Dating - Needs Validation

2nd Round Storyboards - Proof of Concept

Service Blueprint

   The color orange designates our new system


The color orange designates our new system

3. Hi-fidelity prototyping


We created wireframes in Illustrator, and used InVision to create a click-through version. We tested these initial wireframes with a couple radiologists before moving into a code prototype.

Usability Testing

After creating a code prototype using AngularJS, we ran usability testing with 4 radiologists. I created task prompts and ran four think-aloud tests. 


4. Designing final features

5. Final Report

For more details, check out our final report: