Inside the Virta App
Sarah Romoslawski
Mixed Methods UX Research and Leadership
Patient Expectations for Health Assessments
Research process for concept testing new features in a telemedicine application
Virta is a digital therapeutic for reversal of Type 2 Diabetes
About Virta
dietary change (low carb, keto)
lifestyle change
health coach
health care provider (MD, NP)
log biomarkers daily (weight, blood glucose, ketones)
Ask: How can we incorporate whole-person health assessments that encourage patients to engage?
Goal: Capture user data and in turn provided a feedback loop to the patient. Create both short term value (setting and achieving small goals) and long term value (personalized treatment towards reversal).
Research Design and Proposal
Gathering Hypotheses with a Multifunctional Team
a psychologist with expertise in behavioral health
product designer
product manager
2 health coaches
software developer
user researcher (myself)
Questions we had:
What goes through a patient's mind if the app asks them to give further data?
How do we establish trust when collecting data?
What feelings or emotions do these assessments bring up?
Do these types of assessments cause introspection?
How can we roll out the question content so that patients are more likely to engage?
Would patients want to work with their coach on these topics? In what ways would they expect to do so?
Does time in treatment impact all of the above? If so, how do we design for newer patients vs. patients further along?
Research questions should guide methodology, recruitment and execution
Finding Alignment
Research Questions
Business goals
User cohorts and their particular needs (if we know them)
Assumptions and Hypotheses
Screenshots of adjacent products, competitor products, or other inspiration if applicable
Methodology recommended
Number of participants
Budget
Timeline
Creating Script and Stimuli for a Remote, Moderated Concept Test
Google doc version of the incomplete script
This mock up is derivative of sketches and notes we made with the psychologist while working on the script
Creating the Script
We iterated on the script in tandem with the designer, who created just enough stimuli to bring the study to life
Collaborating with the psychologist on the project, we worked screen by screen with the knowledge of what content would be in place
We avoided waiting for a perfectly polished set of mockups
Biomarker vs. Task Card
One variation of the script showing one of the possible onboarding solutions for the assessment feature
Goal: Uncover Where Feature Lives and How Patients Might Take Action
"Where would you go to find this?"
"Where would you go later to find if you had not completed it yet?"
"Does this meet your expectations?"
Solution: Create two versions of the script
Conducting the Remote, Moderated Concept Test
Example email for participant recruitment via Survey Monkey
Recruiting for the Study
We sent the survey out to two groups:
Patients who were early in the treatment
Patients who were past 90 days into the treatment
The recruitment survey asked about the following:
Interest in participation after reviewing the details
Satisfaction with the Virta treatment
Availabilities
Access to the internet and the ability to view a shared screen
Willingness to sign a consent to record legal form
Final participant count:
9
90 Days or more
in treatment
in treatment
9
30 Days or less
in treatment
in treatment
18
Total Participants
Notes made after the first pilot to adjust questions during wrap up
Pilot Study
Virta Ambassador
Assess timing
Find gaps in questions
Make changes to script, timing, follow up questions, etc...
Note-taking sheet provided to observers
Interview Observation and Note Taking
Open doors to involvement
Provide note taking template
Use separate phone booths
Introduce everyone on the call to the participant so he or she knows who is on the line (Consent is given prior to the study in a legal form)
VP Product, engineers, health coaches, enrollment, marketing, design, facilities
Expedite analysis
Analysis and Findings
Restate the research questions
Summarize findings at the front for those that need to revisit the report quickly
Provide evidence in the body of the report for those that want to go deeper
Make recommendations based on evidence
Findings and Recommendations
Virta is still iterating on this work and thus any detailed findings are still confidential. On a high level, we learned the following:
Assumption invalidated! People want to work on their behavioral health and while doing so they want to feel a sense of autonomy
Without enough transparency on what happens behind the scenes after the assessment, patients will not feel agency in their actions
Patients expect their coach and provider to be proactive in setting goals with them
The assessments can be messaged and labeled in a more light-hearted and introspective way
Introspection seems to be important to patients so that they can take action on their progress