Analysis of Case Studies' Contribution to Improving the Quality of Health Risk Communication

Analysis of the 10 case studies reveals the varying perspectives of PHS agencies on the content, implementation, and expected outcomes of health communications strategies. A combination of quantitative and qualitative analytic techniques was used by the Subcommittee in comparing PHS health risk communication practices against a model standard of risk communication developed by EPA. Frequency counts of EPA's Cardinal Rule critical elements (Appendix 1) were tabulated on both a cross-case and individual-case basis. Second-order numbers, such as means and percentage distributions, were calculated to examine the tabulations and their relationships further. Results of that analysis are shown in Tables 2 through 4.

Findings from Table 2 show a range of variation as well as some consistency in how effectively PHS agencies planned, implemented, and evaluated health communication strategies. Some of the lower percentage values (i.e., 14 percent and 23 percent) attributed to a couple of the case studies were due in part to a lack of sufficient information. Without the necessary information, the Subcommittee was unable to make precise judgments or statements about health risk communication effectiveness.

With those few exceptions, enough information and data were available to show that most PHS agencies recognized the importance of and need for a systematic approach to health risk communications.

Two case studies in particular serve as prime examples of how agencies benefit when health communication is well planned and well executed. In an effort to inform the public about the risks and benefits of fluoride, EHPC identified several key factors that contributed to the overall success of its health communication campaign. These include (1) having a sense of timing, or knowing when to release information to achieve maximum public impact; (2) emphasizing the importance of message clarity, balance, and accuracy when designing and sending health communication messages; and (3) having the commitment of the organization's decision-makers and management, which can enhance the effectiveness of the health risk communication effort.

Another clear example of how health risk communication methods can work came from FDA. With a few effective principles, FDA was able to reach its intended audience with the message about the dangers of defective heart valves. Contributing to FDA's health communication success were the use of focus groups, enrollment incentives to join a heart valve registry, careful message design and clarification, and solicitation of health professionals' responses to notification messages.

Table 4 takes the analysis of results in Table 3 one step further, to look more specifically at the Cardinal Rules that figured prominently in health communication activities of PHS agencies. On the basis of percentage distributions, the Subcommittee was able to group the Cardinal Rules for each case into the three broad categories of "highly effective," "moderately effective," and "least effective." A frequency count of the Cardinal Rule critical elements provides a revealing set of general characteristics associated with health risk communication effectiveness. An analysis of the data in Table 4 reveals that in the "highly effective" category, Cardinal Rules 1 (Accept and involve the public as a legitimate partner) and 5 (Coordinate and collaborate with other credible sources) seem to be common to most PHS agencies.

In the "moderately effective" category, Rule 6 (Meet the needs of the media) is found most frequently. In the final category of "least effective," Rule 2 (Plan carefully and evaluate efforts) occurs most often.

A qualitative analysis of the case studies reveals varying beliefs among PHS agencies about what the content, process, and outcomes of a health risk communication should be. To simplify the analysis, the Subcommittee grouped the EPA Cardinal Rule critical elements into the broadly recognized communication categories shown previously in Table 1: communication content, communication process, and evaluation of communication outcomes and impacts. Using this schema, the Subcommittee was able to identify some perceived strengths and weaknesses of PHS health risk communication practices.

Communication content, the first component, refers to the value of the information or message as perceived by the receiver. The amount of emphasis given to content varied substantially among the agencies. FDA, for example, expended a great amount of effort in content analysis and the pretesting of messages. Some agencies, on the other hand, chose to create messages with little audience input. Gaps in the content component of health communication campaigns were fairly evenly distributed across the agencies and present some relevant questions:

1. Did content facilitate the achievement of communication objectives? If not, why?
   
2. Was message and information content relevant to the targeted behavior (i.e., attitude and behavior change)?
   
3. Did the audience understand the information; was it relevant and useful?

Communication process is the second and most labor-intensive of the components. During this stage, the rationale for choosing specific health risk communication strategies and techniques is proven either effective or ineffective. Most of the agencies that submitted case studies are currently involved in health risk communication activities to some extent. Yet, few of the agencies could clearly explain the rationale and content of their health risk communication goals and strategies. Several factors contributed to this condition. First is an inadequate recognition that audiences differ greatly in value systems and levels of involvement in the health risk communication process. FDA provided one exception to this trend by conducting focus tests with their audiences to identify relevant needs and expectations. Second, systematic methods for defining health risk communication needs and responsibilities were unspecified. Third, discussion about the role of health risk communications as a function of agency mission, goals, and objectives is too limited. Finally, barriers to the effectiveness of health risk communications may be internal, such as constraints of staff, resources, and budget, or external, such as the activities of interest groups or the limits set by policy or mandate.

Having to explain health risk and uncertainty were common experiences shared by most of the agencies. Yet the specific procedures that were used in the various risk events differed according to the type of health risk issue, the scope of the problem, and levels of public concern. CDC, in its Hanford Environmental Dose Reconstruction Project, provides a excellent example of how, when properly implemented, the process can achieve the desired outcomes. CDC very effectively organized its health risk communication campaign around the simple principle of using multiple media strategies to build community trust and support.

The last component, outcome and impact evaluation, is an ongoing and systematic procedure for assessing the efficacy of health risk communication strategies in achieving intended outcomes. Evaluation, among the agencies, was the least understood of the communications components. Collecting process and anecdotal information was the preferred method of evaluation. However, to properly judge the effect of a health risk communication activity, measurable objectives must be designed in the planning stage and tracked until the completion of the activity.

One of the evaluation successes involved NIMH and its study of reducing sexual risk behaviors among runaways. Basic evaluation rules were put in place before, during, and after the intervention program. Both qualitative and quantitative evaluation measures were used. With these measures, NIMH was able to assess audience perceptions, track the achievement of communication objectives, and improve the quality of services provided to runaways.

TABLE 2. Percentage Distribution of EPA Cardinal Rule Elements Used Within Each PHS Agency's Health Risk Communication Activities

TABLE 3. Mean and Percentage Distribution of EPA Cardinal Rule Critical Elements Used Across PHS Agency's Health Risk Communication Activities

TABLE 4. Effectiveness of PHS Health Risk Communication Practices in Following EPA's Seven Cardinal Rules of Risk Communication