Early efforts. In 1993, the Public Health Service (PHS) undertook an analysis of risk communication policies and practices across its agencies with the goal of developing recommendations on improving health risk communication. The Environmental Health Policy Committee (EHPC) charged its subcommittee on risk communication and education with the analysis. Member agencies of the subcommittee submitted examples of health risk communication activities or decisions they perceived to be effective and examples of cases they thought had been less effective. Information contained in the case studies was compared with the U.S. Environmental Protection Agency's Seven Cardinal Rules of Risk Communication. A major finding of the study was that the PHS health risk communication performance was least effective in practicing Cardinal Rule 2 (plan carefully and evaluate efforts). Among the agencies, evaluation was the least understood of the communication components, which also include message content and delivery. Collecting process and anecdotal information was the preferred method of evaluation, with relatively little or no emphasis on outcome and impact evaluation.
The subcommittee identified evaluation as a critical area for improvement, and issued specific recommendations aimed at addressing the evaluation needs of various PHS agencies. One recommendation was to organize an interagency initiative aimed at increasing awareness and visibility of pressing health risk communication issues such as evaluation. A second recommendation was for each PHS agency to consider developing a set of generally accepted practices and guidelines for effective evaluation of communication activities and programs.
Issue to be addressed. In response to these two recommendations, the subcommittee formed a workgroup to organize a 1day workshop to assist PHS decisionmakers and health risk communication practitioners in improving their effectiveness in evaluating health risk messages and information. The workshop was held on June 9, 1994, in Annapolis, Maryland. A total of 38 health risk communication practitioners from various PHS and other Federal agencies attended the workshop, which was entitled Applied Evaluation Methods for Health Risk Communication. The workshop participants were selected because of their responsibility for and commitment to effective health risk communication practice at the Federal level. The Federal agencies and organizations who participated in the workshop are listed in Appendix 1.
The two presentations and five case studies examined (1) what methodologies and techniques PHS and other Federal agencies used to evaluate specific health risk communication campaigns, and (2) the philosophical underpinnings of evaluation for policy and program decisions.
Key findings. The case studies demonstrate how health risk communication planning with appropriate means of evaluation (1) is an integral part of any comprehensive communications program to promote health and prevent disease; (2) increases the likelihood that health risk communication strategies will achieve their stated objectives; and (3) reduces the likelihood that the potential benefits of public health programs are delayed, diminished, or lost because of poorly targeted or misdirected messages.
Formative evaluation methods were a major focus of all five agencies' evaluation efforts. A combination of formative evaluation techniques, such as focus groups, questionnaires, and interviews, were used for defining the purpose of the message(s); identifying appropriate target audiences; determining what message ideas or concepts have the best chance of connecting with the target audience; and selecting distribution channels. The result was that agencies were better able to assess if messages, materials, and other communications products were
Using innovative evaluation techniques represents a critical step toward improving the way PHS agencies conceive, develop, and deliver health risk communication messages and materials.
Return to Table of Contents
Return to Committee Reports Page