In the following sections, we describe our methodological approach to these tasks.Reviewing Background Materials
To assure that our working assumptions appropriately reflected the prior work of the Objective 11-4 Technical Expert Workgroup, we reviewed background materials the ODPHP Project Officer provided:
We also examined the standards and protocols produced by the two organizations that have developed processes to review and assess the quality of health websites, to assure that our definitions of terms, identified disclosure elements and criteria, and language were consistent with those of these established and recognized quality initiatives. These included:
This review assured that the assumptions guiding our work and our definitions of terms and concepts were consistent with evolving standards in the field.Defining the Denominator
Several assumptions guided our definition of the universe of health websites on the Internet (the "denominator") and the sampling strategy for this project. First, we assumed that many (if not most) Internet users search for health information using standard search engines, such as Yahoo or Google, and then visit several different websites identified. Second, we assumed that topics of interest, and therefore search terms, will vary, depending not only on individual interests, but also on health issues in the news. Thus, while a small number of heavily used consumer-oriented health information websites may address common areas of interest or serve as portals to specialized sources of information, any given search might in fact lead the consumer to a large number of websites designed for many different audiences and many different purposes. We therefore defined "health-related website" in inclusive terms consistent with those suggested by URAC, Consumer Health WebWatch, and the eHealth Code of Ethics, to include websites associated with a wide variety of sponsoring organizations that provide information for staying well, for preventing and managing disease, and for making decisions related to health, health care, health products, or health services.3
For the purpose of defining the denominator, we sought to enumerate an inclusive universe of health websites visited by Internet users. We therefore relied on information generated by a commercial vendor (Hitwise) that tracks Internet traffic, in lieu of smaller or more selective lists available from other sources. Hitwise uses information from Internet service providers to monitor traffic and categorizes websites with a minimum number of visits by market type, based on their subject matter and content4. Market share is then reported based on the number of visits to each website within the market, the number of pages viewed per visit, and the amount of time spent per visit.
We purchased data from Hitwise on websites classified in its Health and Medical market category, based on Internet traffic in the United States during October 20055. Although information was also available on subgroups within this market (including the Health Information subgroup), we defined the "universe" for this project as all 3,608 websites within the overall Health and Medical category, in order not to exclude websites that might contain health information consistent with our working definition. Of these 3,608 websites, 213 accounted for 60 percent of all visits within this market. The dataset we purchased from Hitwise included the following information for all websites: domain name, ranking (from 1 to 3,608) within the Health and Medical category (based on number of visits), percentage of market share by visits, percentage of market share by number of page views, and average length of each visit (expressed in minutes: seconds). We used market share defined by number of visits to identify the heavily trafficked sites, since page views and the average length of visits could reflect navigational difficulties rather than user interest. Hitwise also provided domain names and limited information on market share of the top 100 sites within each of 11 subcategories within the Health and Medical category6. We used this information to assure diversity in our sample.
3 The e-Health Code of Ethics defines "health information" as follows: "Health information includes information for staying well, preventing and managing disease, and making other decisions related to health and health care. It includes information for making decisions about health products and health services. It may be in the form of data, text, audio, and/or video. It may involve enhancements through programming and interactivity." Accessed June 19, 2005 at [www.estrategy.com]. This definition was also adopted by Risk, A., and Dzenowagis, J. "Review of Internet Health Information Quality Initiatives." Journal of Medical Internet Research, vol. 3, no. 4, 2001.
4 Hitwise monitors visits to primary domain names or sub-domain names only (that is, the initial part of a URL address up until the first slash). Such URL addresses typically direct users to a website's homepage. Hitwise does not track visits to more specific websites within a domain or sub-domain.
5 Hitwise excluded from its listing any websites that accounted for less than 0.01 percent of Internet traffic within the Health and Medical category during October 2005.
6 Hitwise also reports information on subcategories within each of its larger market categories. The 11 subcategories in the Health and Medical category are Information, Research, Well-being, Primary and Specialist, Pharmaceutical, Pharmacies, Paramedical, Organization, Health Insurance, Hospitals, and Alternative Medicine.