Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact.
(73/6835)
The question of why to evaluate a programme is seldom discussed in the literature. The present paper argues that the answer to this question is essential for choosing an appropriate evaluation design. The discussion is centered on summative evaluations of large-scale programme effectiveness, drawing upon examples from the fields of health and nutrition but the findings may be applicable to other subject areas. The main objective of an evaluation is to influence decisions. How complex and precise the evaluation must be depends on who the decision maker is and on what types of decisions will be taken as a consequence of the findings. Different decision makers demand not only different types of information but also vary in their requirements of how informative and precise the findings must be. Both complex and simple evaluations, however, should be equally rigorous in relating the design to the decisions. Based on the types of decisions that may be taken, a framework is proposed for deciding upon appropriate evaluation designs. Its first axis concerns the indicators of interest, whether these refer to provision or utilization of services, coverage or impact measures. The second axis refers to the type of inference to be made, whether this is a statement of adequacy, plausibility or probability. In addition to the above framework, other factors affect the choice of an evaluation design, including the efficacy of the intervention, the field of knowledge, timing and costs. Regarding the latter, decision makers should be made aware that evaluation costs increase rapidly with complexity so that often a compromise must be reached. Examples are given of how to use the two classification axes, as well as these additional factors, for helping decision makers and evaluators translate the need for evaluation--the why--into the appropriate design--the how. (+info)
A mobile unit: an effective service for cervical cancer screening among rural Thai women.
(74/6835)
BACKGROUND: We carried out a systematic screening programme using a mobile unit with the purpose of increasing use of Papanicolaou (Pap) smear screening among rural Thai women. The mobile unit campaign was carried out initially between January and February 1993 and then in 1996 in all the 54 rural villages in Mae Sot District, Tak Province, northern Thailand. METHODS: To evaluate the effect of the programme on changes in knowledge and use of screening, we compared the results of three interview surveys of women, 18-65 years old, in villages selected by systematic sampling for each survey; first in 1991 (before the operation of the programme), secondly in 1994 (one year after the first screening campaign), and last in 1997 (one year after the second campaign). This report also compares data on Pap smears taken by the mobile unit with other existing screening services in the study area. RESULTS: A total of 1603, 1369, and 1576 women respectively, participated in each survey. The proportion of women reported knowing of the Pap smear test increased from 20.8% in 1991 to 57.3% in 1994 and to 75.5% in 1997. The proportion of women who had ever had a Pap smear increased from 19.9% in 1991 to 58.1% in 1994 and to 70.1% by 1997. Screening by the mobile unit accounted for 85.2% of all cervical intraepithelial neoplasia (CIN) III and all invasive cancers identified among the Pap smears taken by screening services in the area between 1992 and 1996. The rate of CIN III was 3.5/1000 smears in this screening programme, which was 5.2 and 2.0 times higher than the rates in the maternal and child health/family planning clinic and the annual one-week mass screening campaign respectively. CONCLUSIONS: The use of a mobile unit may be an effective screening programme in rural areas where existing screening activities cannot effectively reach the female population at risk. (+info)
Tuberculosis epidemiology and control in Veracruz, Mexico.
(75/6835)
BACKGROUND: Tuberculosis (TB) rates remain high in regions of Southern Mexico despite the existence of a National Tuberculosis Program. Understanding TB epidemiology in such settings would assist in the design of improved TB control and highlight the challenges confronting TB control in developing countries. METHODS: We conducted a retrospective review of treatment control cards from 1991 to 1994 in five municipalities in a semiurban region of Southern Mexico. RESULTS: The relatively high rate of TB observed, 42.6 per 100,000 inhabitants, did not change significantly during the study period. Cure rates among new cases were 79% and significantly lower among retreatment cases (62%). Directly observed therapy (DOT) was administered to 84% of patients. Approximately one-half of the retreatment cases who were not cured were compliant with therapy, suggesting that drug resistance contributed to these poor results. Of particular concern was a core group of 16 patients who had received at least three treatments. CONCLUSIONS: This region of Mexico has persistently high TB rates despite a DOT-based TB control programme which achieves an overall cure rate of 77%. There exist many retreatment cases for whom cure rates are significantly lower. These cases may serve as a core group for the dissemination of drug resistant TB. The control programme is being reinforced by a nominal register of patients, decreasing administrative barriers for drug supply to individual patients and the availability of mycobacteria cultures. In addition to these measures, in regions which are approaching the levels of efficacy recommended by the WHO it may be appropriate to consider focusing efforts on the identification and treatment of chronic cases. (+info)
Barriers between guidelines and improved patient care: an analysis of AHCPR's Unstable Angina Clinical Practice Guideline. Agency for Health Care Policy and Research.
(76/6835)
OBJECTIVES: To describe common barriers that limit the effect of guidelines on patient care, with emphasis on recommendations for triage in the Agency for Health Care Policy and Research (AHCPR) Unstable Angina Clinical Practice Guideline. DATA SOURCES: Previously reported results from a prospective clinical study of 10,785 patients presenting to the emergency department (ED) with symptoms suggestive of acute cardiac ischemia. STUDY DESIGN: Design is an analysis of the AHCPR guideline with regard to recognized barriers in guideline implementation. Presentation of hypothetical scenarios to ED physicians was used to determine interrater reliability in applying the guideline to assess risk and to make triage decisions. PRINCIPAL FINDINGS: The AHCPR guideline's triage recommendations demonstrate (1) poor interobserver reliability in interpretation by ED physicians; (2) limited applicability of recommendations for outpatient management (applies to 6 percent of patients presenting to the ED with unstable angina); (3) incomplete specifications of exceptions that may require deviation from guideline recommendations; (4) unexpected effects on medical care by significantly increasing the demand for limited intensive care beds; and (5) unknown effects on patient outcomes. In addition, analysis of the guideline highlights the need to address organizational barriers, such as administrative policies that conflict with guideline recommendations and the need to adapt the guideline to conform to local systems of care. CONCLUSIONS: Careful analysis of guideline attributes, projected effect on medical care, and organizational factors reveal several barriers to successful guideline implementation that should be addressed in the design of future guideline-based interventions. (+info)
Experiences and attitudes of residents and students influence voluntary service with homeless populations.
(77/6835)
OBJECTIVE: To assess the impact of two programs at the University of Pittsburgh, one that requires and one that encourages volunteer activity. In the program that requires primary care interns to spend 15 hours in a homeless clinic, we measured volunteer service after the requirement was fulfilled. In the program that encourages and provides the structure for first- and second-year medical students to volunteer, we assessed correlates of volunteering. MEASUREMENTS AND MAIN RESULTS: When primary care interns were required to spend time at homeless clinics, all (13/13) volunteered to work at the same clinic in subsequent years. Categorical interns without this requirement were less likely to volunteer (24/51; chi2 = 12.7, p >. 001). Medical students who volunteered were more likely to be first-year students, have previously volunteered in a similar setting, have positive attitudes toward caring for indigent patients, and have fewer factors that discouraged them from volunteering (p <. 01 for all) than students who did not volunteer. CONCLUSIONS: Volunteering with underserved communities during medical school and residency is influenced by previous experiences and, among medical students, year in school. Medical schools and residency programs have the opportunity to promote volunteerism and social responsibility through mentoring and curricular initiatives. (+info)
Predicting posttreatment cocaine abstinence for first-time admissions and treatment repeaters.
(78/6835)
OBJECTIVES: This study examined client and program characteristics that predict posttreatment cocaine abstinence among cocaine abusers with different treatment histories. METHODS: Cocaine abusers (n = 507) treated in 18 residential programs were interviewed at intake and 1-year follow-up as part of the nationwide Drug Abuse Treatment Outcome Study (DATOS). Program directors provided the program-level data in a mail survey. We applied the hierarchical linear modeling approach for the analysis. RESULTS: No prior treatment and longer retention in DATOS programs were positive predictors of posttreatment abstinence. The interactive effect of these 2 variables was also significantly positive. Program that offered legal services and included recovering staff increased their clients' likelihood of cocaine abstinence. Crack use at both the client and program level predicted negative impact. None of the program variables assessed differentially affected the outcomes of first-timers and repeaters. CONCLUSIONS: Although treatment repeaters were relatively difficult to treat, their likelihood of achieving abstinence was similar to that of first-timers if they were retained in treatment for a sufficient time. First-timers and repeaters responded similarly to the treatment program characteristics examined. The treatment and policy implications of these findings are discussed. (+info)
Prevention of relapse in women who quit smoking during pregnancy.
(79/6835)
OBJECTIVES: This study is an evaluation of relapse prevention interventions for smokers who quit during pregnancy. METHODS: Pregnant smokers at 2 managed care organizations were randomized to receive a self-help booklet only, prepartum relapse prevention, or prepartum and postpartum relapse prevention. Follow-up surveys were conducted at 28 weeks of pregnancy and at 8 weeks, 6 months, and 12 months postpartum. RESULTS: The pre/post intervention delayed but did not prevent postpartum relapse to smoking. Prevalent abstinence was significantly greater for the pre/post intervention group than for the other groups at 8 weeks (booklet group, 30%; prepartum group, 35%; pre/post group, 39%; P = .02 [different superscripts denote differences at P < .05]) and at 6 months (booklet group, 26%, prepartum group, 24%; pre/post group, 33%; P = .04) postpartum. A nonsignificant reduction in relapse among the pre/post group contributed to differences in prevalent abstinence. There was no difference between the groups in prevalent abstinence at 12 months postpartum. CONCLUSIONS: Relapse prevention interventions may need to be increased in duration and potency to prevent post-partum relapse. (+info)
Interventions to improve the delivery of preventive services in primary care.
(80/6835)
OBJECTIVES: This review was conducted to determine the effectiveness of different interventions to improve the delivery of preventive services in primary care. METHODS: MEDLINE searches and manual searches of 21 scientific journals and the Cochrane Effective Professional and Organization of Care of trials were used to identify relevant studies. Randomized controlled trials and controlled before-and-after studies were included if they focused on interventions designed to improve preventive activities by primary care clinicians. Two researchers independently assessed the quality of the studies and extracted data for use in constructing descriptive overviews. RESULTS: The 58 studies included comprised 86 comparisons between intervention and control groups. Postintervention differences between intervention and control groups varied widely within and across categories of interventions. Most interventions were found to be effective in some studies, but not effective in other studies. CONCLUSIONS: Effective interventions to increase preventive activities in primary care are available. Detailed studies are needed to identify factors that influence the effectiveness of different interventions. (+info)