This page has been archived.

Click here to skip navigationDepartment of Health and Human Services logo

Expanding the Reach and Impact of
Consumer e-Health Tools

June 2006

Office of Disease Prevention and Health Promotion logo

Table of Contents
Executive Summary (Stand-Alone)
Preface: A Vision of e-Health Benefits for All
Executive Summary
Chapter 1. Introduction
Chapter 2. Mapping Diversity to Understand Users’ Requirements for e-Health Tools
Chapter 3. Assessing the Evidence Base for e-Health Tools for Diverse Users
Chapter 4. Strategic Factors in Realizing the Potential of e-Health
Chapter 5. Partnerships for Meaningful Access
Appendix 2. Project Interviewees, Experts Consulted, and Reviewers
Appendix 3. Chapter 3 Literature Review Summary
Appendix 4. A Comparison of Internet Use and Health Status of Populations That Experience Health Disparities

Appendix 1. Environmental Scan of 40 e-Health Tools

Between August 2003 and February 2004, project staff conducted an environmental scan of consumer e-health tools in the academic, nonprofit, and commercial sectors. The scan was based on review of two major e-health research programs (the National Cancer Institute’s [NCI] Small Business Innovation Research [SBIR] program and the Robert Wood Johnson Foundation [RWJF] Health e-Technologies Initiative); articles and citations in the peer-reviewed social science, biomedical, and public health journal literatures; and recommendations from tool developers and other experts in the field. Project staff identified 40 tools for in-depth investigation. The purpose of the scan was to learn about the major characteristics, intended audiences, and evaluation practices of a range of tools. Examples were sought that are recognized by major research funders, use methodological rigor in their evaluations, have public health significance and commercial viability, and/or are technologically innovative. There was no expectation that this exercise would “cover the waterfront” or collect generalizable information. Inclusion in this exercise does not in any way imply an endorsement or evaluation of the quality or effectiveness of the tool.

For consistency, and to glean as much information as possible, the scan was conducted using a standard instrument (see Instrument Used to Conduct Environmental Scan for Consumer e-Health Report). Questions were based on theories, methods, concepts, and terminology from the peer-reviewed literature; reports on the state of e-health technologies; and handbooks on health communication research. The questions were pilot-tested with experienced e-health developers and researchers and revised based on their comments and suggestions. The instrument was used to conduct interviews with e-health tool developers and other experts (see Appendix 2 for names of interviewees) and review the tools themselves. Staff also sought out information on the tools in journal articles and other documents that were either publicly available or supplied by the tool developers.

Information was collected on the following topics, using the form at the end of this appendix:

  • Functionalities of the e-health tool

  • Methods of delivery

  • User groups, populations served, and their effect on design and evaluation

  • Payer(s) for use of the e-health tool

  • Prospective purchasers and stakeholders other than consumers/patients

  • Research and evaluation practices, including data elements collected

  • Privacy, confidentiality, and security practices

  • Mechanisms for dealing with adverse events

The resulting descriptions, presented below, represent a late 2003 and early 2004 snapshot of the e-health phenomenon. That phenomenon is rapidly moving: new technology is routinely being introduced; the market surrounding digital applications is in flux (at least two companies in the interview group were acquired during the short interview phase); and grant cycles begin and end. Nearly all of the interviewees described forthcoming products, services, research, or publications that will change the profile of their tools.

Back to Top

Tool Functions

Function Number offering (of 40)
Health Information
Behavior change
Other (one or more additional functions)
Personal health data entry
Decision support
Social/emotional support
Disease management
Secure provider/patient communication
Risk assessment
Personal health record
All of the e-health tools in this group offer users multiple functions. Counting the “other” category as a single function (which understates the reality), the average tool has more than 5 functions of 10 possible choices. The core function, unsurprisingly, is health information, followed closely by behavior change facilitation. The large number of behavior change/prevention tools is partly accounted for by the presence in this pool of 20 NCI and RWJF grantees. At the time, both of these research programs stressed prevention-oriented projects. Significantly, 24 of the 40 tools offer one or more functions other than the nine specified in the interview form. This reflects the uniqueness and originality of e-health tools. The number of tools offering specific functions is shown, in order of frequency, in the following table.

Back to Top

Delivery Methods

The “average” e-health tool in this group of 40 uses at least two delivery methods—once again treating the “other” category, for simplicity, as representing a single method. In fact, as with functions, the “other” category is large and diverse and includes several unique devices for collecting and transmitting personal health data. The overwhelming number of e-health tools in the interview group—34 of 40—are delivered through the Internet, either through restricted-access (member/subscriber) Web sites, public Web sites, or a combination. Some tools that were initially developed for delivery via CD-ROM, notably, the Comprehensive Health Enhancement Support System (CHESS), have been converted to the Internet. e-Health tools generally use more than one delivery method. However, in most cases a primary form of delivery (e.g., secure, restricted-access Web sites) is combined with one or more ancillary methods (e.g., e-mail notices).

Back to Top

Audiences and Audience Segments

The findings show the complexity of e-health audience variables and the many ways developers think about reaching their intended audiences or user groups. The primary strategy used by developers in this group of 40 is audience segmentation. The findings align with the observations made in the Institute of Medicine report, Speaking of Health, about the adaptation of health communication for diverse audiences (2002):

  • Some tools are developed for narrowly defined audiences (e.g., people older than age 65 with chronic obstructive pulmonary disease [COPD], or binge-drinking college students). Some developers have an array of such specialized tools or modules.

  • Some tools are developed for a broad cross-section of users but are subsequently adapted to serve different audience segments (e.g., a Spanish-language version, a module for pregnant women, a chat room for caregivers). The broad cross-section may exist because the tool is available to all comers (e.g., through a public Internet site) or because it is distributed to a restricted but diverse constituency (e.g., the employees of a distributor or health plan enrollees).

  • Some tools are developed for a broad (and therefore presumably heterogeneous) user group in a way that focuses on what all users have in common.

Back to Top

Transferability of Personal Health Information

The interviewees were asked what would be required for the user to transfer personal health data (e.g., history of tobacco use, blood sugar, blood pressure) to another organization’s application or device. The findings were varied and sometimes ambiguous. Of the 23 tools on which there is information for this question, some respondents focused on users’ ability to get their data in any form, including print, while others focused on interoperability issues related to standards and other technical matters. Only 7 tools have technical interoperability with other electronic systems. Another 7 make users’ data available to them in print. In general, the answers indicate the distance yet to go to make applications interoperable and to provide alternatives to proprietary approaches.

Privacy, Confidentiality, Security, and HIPAA

For these tools, security and confidentiality protections are generally addressed at the design stage, with a monitoring protocol thereafter. All interviewees in this group indicated awareness and, where needed, detailed knowledge of the Health Information Portability and Accountability Act (HIPAA). This fact is tempered by the reality that the HIPAA Privacy Rule does not apply to many e-health tool providers. The interviews highlighted the limits to privacy and confidentiality protection in online communities, as well as participants’ willingness to continue to share despite these limitations. The developers and distributors of open-access e-health tools with chat rooms and listservs make a serious effort to call users’ attention to the fact that the confidentiality of their contributions is not protected; theoretically, consumers use these sites with their “eyes open.” Participants must register, and the chat rooms in both open- and closed-system e-health tools in this group are monitored, and in some cases moderated by trained people, to minimize inappropriate behavior. For the e-health tools that are distributed as part of closed systems (the large majority in this group), chat room and listserv participants’ privacy seems more assured, as a function of the restricted access combined with stringent security measures.

Back to Top

Research and Evaluation

In-house or self-evaluation is the most common form of evaluation, done for 36 of the 40 e-health tools. Nearly one-half (18) are also evaluated by a nonaffiliated third party (i.e., an independent researcher). Only 10 e-health tools are evaluated by an affiliated third party (e.g., a sponsor or purchaser). Two-thirds of the evaluations (26 of 40) use at least one validated measure. All 40 e-health tools have undergone some kind of formative research. Almost all of the e-health tools (36) undergo process evaluation, described as usability testing or “ongoing feedback” (associated with continuing quality improvement). Some form of outcome evaluation has been conducted on the majority of e-health tools (33 of 40), with 17 e-health tools being evaluated in randomized controlled trials. (This is likely an unrepresentatively high proportion and reflects the requirements of the NCI SBIR and RWJF programs.) Many tools have an individual user feedback mechanism, such as a “comments” box or phone line. Developers report using the feedback to modify the tools on an ongoing basis.

The Federal Government emerged as significant, both as a funder of developmental or evaluation research and as a dissemination partner or purchaser of e-health tools. Some of the leading research and development on consumer/patient e-health (notably, on personal health records and disease management) is being done by Government agencies, including the U.S. Department of Veterans Affairs and the Centers for Medicaid & Medicare Services (CMS). In addition, at least 15 of the 40 tools have Government funding (usually research-related), in addition to several that are purchased by Medicare or Medicaid for enrollee use. As noted, several developers indicated that they see CMS as a potential purchaser of their tools. The need for Federal and foundation research funding can also be inferred from the fact that the only tools being rigorously evaluated are those with grant funding. Several interviewees mentioned that they had applied for research funding but did not receive it, and thus were unable to do the desired level of evaluation.

Back to Top

Payers, Purchasers, and Dissemination Partners

Payer/Purchaser Number of tools (of 40)
Government (usually as research support)
Health plans or insurers (includes Medicare and Medicaid)
Healthcare providers
Third-party sponsor (e.g., drug company, device manufacturer)
The questions in the instrument focused on payment for tool development rather than on the mechanisms for dissemination. The information collected shows that most developers in this group have multiple funders or purchasers, and that very few are consumers. Consumers pay to use only 9 of the 40 tools in this group, and of those 9, only 3 tools are exclusively made available directly to consumers (i.e., the tools are also disseminated through intermediaries). The following list shows the number of e-health tools in the interview group that fall in each payer or purchaser category.

The largest number of e-health tool developers (21) say they see health plans or insurers, including Medicare and Medicaid, as “ultimate purchasers or stakeholders” for their products (consumers/patients are always regarded as the “ultimate end-users”). To evaluate and demonstrate their products’ return on investment for purchasers, many tool developers conduct cost-benefit studies to compare health service utilization, absenteeism, or other variables with the cost of distributing the tool.

As noted above, 37 of the 40 tools are disseminated through various dissemination partners, a mechanism used for both for-profit and not-for-profit tools. The partners are in the following categories (with some developers partnering with several):

  • Public health organizations

  • Schools or childcare facilities

  • Healthcare organizations/individual providers

  • Employers

  • Health insurance companies

  • National health advocacy organizations

In these cases, consumers gain access to and experience the tools as a function of their relationship to the distributing entity (e.g., as employees, health plan members, and constituents of a national health organization). Some distribution partners purchase or license the tools and provide them to customers, employees, or members; others distribute the tools as part of healthcare or public health services. Some developers produce both direct-to-consumer and restricted-access versions of their products, with the latter offering more interactive services that are customized to the distribution partner’s specifications.

Instrument Used to Conduct Environmental Scan for Consumer e-Health Report

Name: ____________________________

Date: _____________________________

  1. Sources of information on application or device. (Check all that apply.)

          __  Interview

          __  Web site

          __  Peer-reviewed literature

          __  Self-published report or other non-peer-reviewed document

          __  Other (specify) ____________________________

  1. Bibliographic references available?

          __  Yes

          __  No

  1. Brief description of application available?

          __  Yes

          __  No

  1. Description of the application or device

    1. Application title and URL ____________________________

    2. Developer organization ____________________________

    3. Division or unit ____________________________

    4. Contact name ____________________________

    5. Contact address, e-mail, and phone number


    6. Function of application or device. (Check all that apply.)

      __  Personal health record

      __  Secure provider-patient communication

      __  Health information

      __  Decision support

      __  Social/emotional support

      __  Risk assessment

      __  Behavior change

      __  Disease management

      __  Personal health data

      __  Clinician-entered

      __  Captured by device

      __  Consumer-entered

      __  Other (specify) ____________________________

    1. Method of delivery of application. (Check all that apply.)

      __  Public Web site

      __  Member or subscriber only Web site

      __  CD or DVD

      __  Kiosk

      __  Game console

      __  PDA

      __  E-mail or listserv

      __  Bulletin board

      __  Telephone (any type)

      __  Device other than game or PDA

      __  Other (specify) ____________________________

    1. Intended user group or population served? (Examples: ethnic group, gender, age, income, literacy skills)


    2. Please describe briefly how you take into consideration the characteristics of your intended users in the design and evaluation of your application or device.


    3. Who pays for the use of the application or device? (Check all that apply.)

      __  Consumer/patient

      __  Healthcare provider

      __  Health plan or insurer, including Medicare and Medicaid

      __  Employer

      __  Third-party sponsor, such as a drug company or device manufacturer

      __  Government (as part of access to health care, such as a community health center, or as part of a research project)

      __  Foundation grant

      __  Other (specify) ____________________________

    1. Whom do you think of as the ultimate purchaser(s) or stakeholder(s) of your application or device? (Check all that apply.)

      __  Consumer/patient

      __  Healthcare provider

      __  Health plan or insurer, including Medicare and Medicaid

      __  Third-party sponsor, such as a drug company or device manufacturer

      __  Government (as part of access to health care, such as a community health center, or as part of a research project)

      __  Other (specify) ____________________________

      __  How is this consideration of purchasers and stakeholders reflected in your design and evaluation?

    1. If, as a result of using your application or device, a user creates an electronic history (e.g., tobacco use, blood sugar or blood pressure levels), what would be required for the user to transfer this informationto another organization’s application or device, such as a personal health record? ____________________________

  1. Application or device research and evaluation

    1. Who has conducted/is conducting evaluations of the application or device? (Check all that apply.)

      __  Non-affiliated third party (example: independent researchers)

      __  Affiliated third party (example: sponsor or purchaser of application or device)

      __  In-house or self-evaluation

      __  Other (specify) ____________________________

    1. Does the evaluation use validated measures?

      __  Yes

      __  No

    1. Which types of research and evaluation have you conducted on the application or device? (Check all that apply and please provide a brief description of what you did as part of each type.)

      __  Formative research ____________________________

      __  Process evaluation ____________________________

      __  Outcome evaluation (note data source) ____________________________

      __  Adequacy of confidentiality and security mechanisms ____________________________

    1. On which of the following elements are/were data collected as part of the research and evaluation of the application or device? (Check all that apply.)

      __  Cost-effectiveness for individuals, providers, payers, or sponsoring organizations

      __  Utilization of health services

      __  Frequency of use

      __  Intensity of use

      __  Satisfaction

      __  Convenience

      __  Relevance for users’ needs

      __  User appeal (likability)

      __  Health status change

      __  Attitude or belief change

      __  Knowledge change

      __  Intention change

      __  Behavior change

    1. Please tell us about any other elements that you collect data on as part of the research and evaluation.


    2. Given current concerns about patient safety and adverse events, some people hypothesize that the use of some applications and devices could have unintended, harmful effects. Do you have any mechanism for identifying harmful effects that might occur as a result of using the application or device? ____________________________

    3. Users typically have to provide anywhere from “some” to “a lot” of personal information to use an e-health application or device. Do you assess if your application or device is HIPAA compliant? (Check only one.)

      __  Yes, I’ve done such an assessment.

      __  No, I haven’t done such an assessment.

      __  I have determined that the application or device is exempt and does not require such an assessment.

    1. Can you suggest other developers/researchers you think I should talk to? ____________________________

Back to Top Appendix 2 >