ODPHP commissioned 15 studies over a period of 4 years to
inform the development of an online collection of prevention and wellness
content and to redesign healthfinder.gov. More than 750 people aged 18 to 84
participated in these studies. Special care was taken to recruit and test
adults with limited literacy skills.
Below is an overview of the iterative research process,
followed by a brief description of each study.
Formative Research Phase
- Literature Review
- Web Site (Content) Analysis
- Expert Interviews
- Structured Interviews
- Mental Models Study
- Card-Sorting Study
![Arrow pointing downward.](images/blue_down_arw.gif)
Prototyping Phase
- Prototype Test (Paper and Clickable)
- Usability Studies 1 Through 4
![Arrow pointing downward.](images/blue_down_arw.gif)
Postlaunch Quality Improvement Phase (Ongoing)
- Card-Sorting Study
- Widget Usability Test
- Intermediary Interviews
- Usability Study 5
Back to top
Formative Research Phase
Literature Review
ODPHP conducted a literature review to answer the
following questions:
- Which audience characteristics are the strongest
determinants of homogeneity in terms of seeking prevention information?
- In what ways and to what extent is currently available
prevention information responsive to these patterns?
- Is prevention information that is targeted in content
design and delivery more effective than information that is not targeted?
- How can prevention information be presented effectively
online?
The literature generally supported the finding that
tailored and targeted materials that respond to individual characteristics,
such as readiness to change, can be effective in engaging interest and leading
to health behavior change. The review also found that interactive information
is more engaging and motivating than information that is not interactive.
Web Site (Content) Analysis
ODPHP reviewed content on nine prevention topics available
from four leading health Web sites: American Cancer Society, Mayo Clinic,
WebMD, and Family Doctor. Based on this review, the authors made several
general recommendations for designing health information for Web sites,
including:
- Keep content at a reasonable length.
- Use headings and subheadings.
- Use hyperlinks so users can explore relevant
content.
- Provide interactive tools as appropriate.
- Convey information using graphics or visual
displays.
Expert Interviews
Subject-matter experts in nine health content areas
identified common questions asked by members of the public. The goal was to
identify audience segments based on motivations for seeking information. The
audience segments developed from the expert interviews included:
- Segment 1: Those seeking information about a health
problem for themselves or someone they know.
- Segment 2: Those wanting to find out whether they or
someone they know has a health problem or reason to be concerned about a health
problem.
- Segment 3: Those seeking information to help prevent
the onset of health problems.
Structured Interviews
ODPHP conducted structured interviews with 200 diverse
participants between the ages of 18 and 65 to test the audience segmentation
strategy mentioned above. Examples of key findings include:
- The audience segmentation strategy was validated
(significant differences were found in motives/preferences between the three
segments).
- Respondents did not remain in a particular audience
segment over time.
- Content should reflect the needs/motives of each
segment.
Mental Models Study
Indepth interviews were conducted with 35 English-speaking
adults, aged 18 to 65, with limited health literacy. The primary intent of this
research was to determine how people naturally group disease prevention topics
(i.e., their "mental models") to inform the development of a prevention
prototype. Examples of key findings include:
- Participants grouped the topics of nutrition, obesity,
and physical activity together.
- Other topics were commonly stand-alone concepts, such
as preventing falls and getting flu shots.
- Participants associated the topics of talking to kids
about smoking and talking to kids about substance abuse.
Card-Sorting Study
Eighty-one diverse participants completed the card-sorting
study. As a result of the study, ODPHP created a prioritized list of the most
useful types of content across the three audience segments. These content types
emerged as generally "useful" and "important":
- Basics I need to know (Understanding)
- I would like to learn more (Assessment)
- I can do this (Overcoming Barriers)
- How will this help me? (Motivators)
- Ways I can take action (Strategies)
- Where can I go for help? (Community Resources)
Back to top
Prototyping Phase
Prototype Test (Paper and Clickable)
ODPHP developed and tested a prevention information
prototype with a diverse, nationwide sample of 300 adults. Participants
evaluated the prototype and prevention content on measures of engagement,
self-efficacy, acceptability, and applicability. Examples of key findings
include:
- Participants found the content acceptable (relevant and
useful), but did not rate it as highly for inspiring self-efficacy.
- Qualitative data showed participants' preference for
the following content characteristics:
- Informal, clear, and concise writing
- "Tabbed" organization
- Actionable information with a small-steps
approach
- nteractive tools
- Graphics that aid understanding
- An interface that allows users to "drill down"
through related chunks of information
Usability Study 1
ODPHP performed an initial usability study with 40 adult
women with limited health literacy skills recruited from federally qualified
community health centers in Baltimore, MD. The study was designed to observe
and record any problems encountered by users as they navigated the prototype,
to learn whether changes to the content and interface design would positively
affect self-efficacy measures, and to gauge levels of understanding and
engagement. Examples of key findings include:
- Participants particularly liked checklists, lists of
questions to ask a doctor, access to related resources, graphics that aid
understanding, and options to print.
- The tabbed approach helped orient participants to where
they were in the content.
- Participants found the interactive tools (e.g.,
calculators, quizzes, menu planners) useful because they were immediate,
relevant, and in context.
- Providing small steps for taking action helped improve
participants' self-efficacy.
- Content organization of "What is the behavior?"
(basics), "Why is it important to me/relevance?" (benefits), and "What do I do
about it?" (take action) was logical to participants.
Usability Study 2
ODPHP conducted a second usability test with 13 adult
women with limited health literacy recruited from federally qualified community
health centers in Baltimore, MD. This test was performed to validate changes
made from the prior usability test (Usability Study 1) and to continue to test
levels of engagement, self-efficacy, and understanding. Examples of key
findings include:
- Participants had difficulty using the search function
and often clicked on the "Search" button without entering a search term.
- Most users expected to see a list of topics in an "A to
Z" format.
- When using the myhealthfinder tool, participants did
not easily connect the information they entered into the tool with the search
results they received.
- Tab navigation needs to be more clearly delineated.
- Participants preferred photographs of "real" people to
icons.
- Participants noticed content at the center of the page
more than top, bottom, and side content.
Usability Study 3
ODPHP performed a third usability test with seven diverse
participants recruited in Knoxville, TN. This study provided an overall test of
the new healthfinder.gov site and of the changes made based on previous
testing. Examples of key findings include:
- Most participants used the A to Z section, rather than
the search box, to browse.
- When prompted to use the search function, participants
had little trouble and were able to scan the results page easily.
- Participants were able to navigate easily back to the
home page through strong lefthand navigation.
- Participants particularly liked the photos of "real"
people and the "take action" content.
Usability Study 4 (myhealthfinder)
ODPHP conducted a usability test on the first iteration of
the myhealthfinder tool with 15 diverse participants. Examples of key findings
include:
- Participants had difficulty signing in. Many attempted
to sign in even though they had never created an account.
- Many participants had difficulty creating an account.
For example, they did not distinguish between required and optional fields, had
difficulty creating a unique username and password, and did not understand
error messages.
- Participants did not want to enter personal information
(e.g., height, weight, smoking status).
- Participants often skipped over context and welcome
information.
- Participants needed more than just recommendations;
they wanted corresponding "how-to" information.
Back to top
Postlaunch Quality Improvement Phase (Ongoing)
Card-Sorting Study
ODPHP conducted a card-sorting activity with 30 adults
with limited health literacy recruited through the WIC program (Special
Supplemental Nutrition Program for Women, Infants, and Children) in Alexandria,
VA, and from a federally qualified community health center in Baltimore, MD.
During each of four focus group sessions, participants completed two card sorts
to refine the organization of the Quick Guide to Healthy Living topic pages and
to develop categories and labels for a new Quick Guide landing page. Examples
of key findings include:
- Participants had difficulty understanding the Benefits
tab and tended to organize all information in the Basics and Take Action
categories.
- Participants were very interested in one-page,
printable action-oriented topics and tools using bullet points instead of
paragraphs.
- Participants preferred topics to be placed under
multiple categories.
- Participants tended to like categories of information
labeled for certain groups (e.g., women, older adults, parents).
Widget Usability Test (Be Active Your Way) (Two
Rounds)
ODPHP held two usability tests with 18 participants
recruited from adult literacy programs in Washington, DC. These studies
specifically focused on the usability, usefulness, and understandability of a
widget providing targeted physical activity tips. Examples of key findings
include:
- Participants did not notice the widget on the left and
right margins.
- Many participants thought the sidebar graphic (widget
icon) might be an ad.
- Participants didn't respond well to the word
"barriers."
- Participants often glossed over difficult words, such
as "moderate" and "vigorous."
- Participants often ignored or skipped over very short
words like "is" and "in."
- Participants wanted to use the "Back" button rather
than other navigational elements.
Intermediary Interviews
ODPHP conducted interviews with 10 health intermediaries
recruited from WIC programs and federally qualified community health centers.
The goal of the interviews was to obtain feedback on how healthfinder.gov
content could better serve WIC and community health center clients. Examples of
key findings include:
- Participants thought the organizational structure of
the content (Basics, Benefits, Take Action) was logical and would help them
explain prevention behaviors to their clients.
- Participants cautioned that content should reflect the
cultural sensitivities and economic realities of priority audiences.
- Participants liked the use of positive, empowering
language.
- Participants suggested an increase in the use of images
and symbols to make points "stick."
Usability Study 5
ODPHP conducted a fifth usability test with nine
participants with limited health literacy recruited through two community adult
learning centers in Columbia, MD, to provide feedback on a new Quick Guide
landing page and topic structure. Examples of key findings include:
- Participants often missed items on the right side.
- Participants struggled with scrolling or didn't scroll
at all.
- Participants had high success rates in finding topics
through the Quick Guide organizational structure.
- Participants easily noticed and used the lefthand
navigation.
- Participants needed everything that looked "clickable"
to be "clickable" and wanted navigational elements to be big and easy to
notice.