Evaluation of LabRespond, a new automated validation system for clinical laboratory test results.
(17/376)
BACKGROUND: Manual validation of laboratory test results is time-consuming, creating a demand for expert systems to automate this process. We have started to set up the program "LabRespond", which covers five validation levels: administrative, technical, sample, patient, and clinical validation. We present the evaluation of a prototype of an automated patient validation system based on statistical methods, in contrast to the commercially available program "VALAB", a rule-based automated validation system. METHODS: In the present study, 163 willfully altered, erroneous test results out of 5421 were submitted for validation to LabRespond, VALAB, and to a group of clinical chemists (n = 9) who validated these test results manually. The test results rejected by three or more clinical chemists (n = 281) served as a secondary reference standard. RESULTS: The error recovery rates of clinical chemists ranged from 23.9% to 71.2%. The recovery rates of LabRespond and VALAB were 77.9% and 71.8%, respectively (difference not significant). The false-positive rates were 82.7% for LabRespond, 83.6% for VALAB, and 27.8-86.7% for clinical chemists. Using the consensus of three or more clinical chemists as the secondary reference standard, we found error recovery rates of 64.8% for LabRespond and 72.2% for VALAB (P = 0.06). Compared with VALAB, LabRespond detected more (P = 0.003) erroneous test results of the type that were changed from abnormal to normal. CONCLUSIONS: The statistical plausibility check used by LabRespond offers a promising automated validation method with a higher error recovery rate than the clinical chemists participating in this study, and a performance comparable to VALAB. (+info)
A Bayesian network for mammography.
(18/376)
The interpretation of a mammogram and decisions based on it involve reasoning and management of uncertainty. The wide variation of training and practice among radiologists results in significant variability in screening performance with attendant cost and efficacy consequences. We have created a Bayesian belief network to integrate the findings on a mammogram, based on the standardized lexicon developed for mammography, the Breast Imaging Reporting And Data System (BI-RADS). Our goal in creating this network is to explore the probabilistic underpinnings of this lexicon as well as standardize mammographic decision-making to the level of expert knowledge. (+info)
New approaches to measuring the performance of programs that generate differential diagnoses using ROC curves and other metrics.
(19/376)
INTRODUCTION: Evaluation of computer programs which generate multiple diagnoses can be hampered by a lack of effective, well recognized performance metrics. We have developed a method to calculate mean sensitivity and specificity for multiple diagnoses and generate ROC curves. METHODS: Data came from a clinical evaluation of the Heart Disease Program (HDP). Sensitivity, specificity, positive and negative predictive value (PPV, NPV) were calculated for each diagnosis type in the study. A weighted mean of overall sensitivity and specificity was derived and used to create an ROC curve. Alternative metrics Comprehensiveness and Relevance were calculated for each case and compared to the other measures. RESULTS: Weighted mean sensitivity closely matched Comprehensiveness and mean PPV matched Relevance. Plotting the Physician's sensitivity and specificity on the ROC curve showed that their discrimination was similar to the HDP but sensitivity was significantly lower. CONCLUSIONS: These metrics give a clear picture of a program's diagnostic performance and allow straightforward comparison between different programs and different studies. (+info)
Using scenarios in chronic disease management guidelines for primary care.
(20/376)
The Prodigy system is a guideline-based decision-support system designed to assist general practitioners in England choose the appropriate therapeutic action for their patients. As part of the system, we developed a novel model for encoding clinical guidelines for managing patients with chronic diseases such as asthma and hypertension. The model structures a guideline as a set of choices to be made by the clinician. It models patient scenarios which drive decision making and are used to synchronize the management of a patient with guideline recommendations. The model is robust with respect to available input data and leaves the control of decision-making to the clinician. We have built execution engines to verify the computability of the model. We intend to test the model integrated in up to 200 live systems from at least four system vendors in English General practice. (+info)
Implausible birth weight for gestational age.
(21/376)
Various rules have been proposed to identify and exclude live births with implausible values of birth weight for gestational age from large perinatal data sets. The authors carried out a preliminary evaluation of common rules by examining the frequency and nature of rule-based exclusions among live births in Canada (excluding Ontario) between 1992 and 1994. There were 625 (0.09%), 133 (0.02%), 170 (0.02%), and 2,858 (0.40%) live births identified for exclusion by a median birth weight for gestational age +/-4 standard deviations (SD) rule, a +/-5 SD rule, a rule based on expert clinical opinion, and a modification of Tukey's rule, respectively. The birth weight and gestational age distribution of the exclusions depended on the particular rule used; for example, 12.1% and 0.3% of live births of > or =4,500 g were excluded under Tukey's rule and the rule based on expert opinion, respectively. Infant mortality rates among those excluded were 8-13 times higher than among all live births. Current rules for identifying implausible birth weight for gestational age tend to flag live births at high risk for infant death. Such rules may erroneously attenuate temporal trends in important perinatal outcomes. (+info)
Automated assessment of dipyridamole 201Tl myocardial SPECT perfusion scintigraphy by case-based reasoning.
(22/376)
This study evaluated the diagnostic accuracy of case-based reasoning (CBR) to automatically detect significant coronary artery disease from dipyridamole 201Tl myocardial SPECT perfusion scintigrams. METHODS: The study population included 240 patients (182 men, 58 women; mean age +/- SD, 61 +/- 12 y) on whom coronary angiography and perfusion scintigraphy were performed within 6 +/- 11 d of each other. The patients were divided into two groups according to the presence or absence of significant coronary disease in any major coronary vessel. Regional myocardial tracer uptake was observed in 84 segments by polar map analysis. For each scintigraphic image, a CBR algorithm based on a similarity metric was used to identify similar scintigraphic images within the case library. The angiographic results of these similar cases were used to obtain the CBR reading, which was compared with the true angiographic results. Myocardial scintigrams were also analyzed by a first-generation Cedars-Sinai (CS) method, including a comparison with a reference database, and by the visual analysis of an expert reader. RESULTS: By receiver-operating-characteristic analysis, the diagnostic accuracy of CBR was not different from the interpretation by the CS algorithm and from visual interpretation (P = not significant [NS]). For detection of significant coronary disease, the respective sensitivities at 50% and 80% specificity were 90% and 67% for CBR, 88% and 65% for CS polar map analysis, and 91% and 74% for visual interpretation. For the detection of coronary disease in the vascular territories assigned to the left anterior descending and the right coronary arteries, CBR and CS polar map analysis showed similar diagnostic accuracy (P = NS). However, for detection of disease in the circumflex artery, CS polar map analysis was slightly better than CBR (P = 0.03). CONCLUSION: Automated interpretation of dipyridamole 201Tl myocardial SPECT perfusion images by CBR has diagnostic accuracy similar to that of visual interpretation or CS analysis. Thus, use of a case library that includes a variety of normal and abnormal perfusion images does not appear to have greater diagnostic power than use of reference limits. (+info)
EASY--an Expert Analysis SYstem for interpreting database search outputs.
(23/376)
With the ever-increasing need to handle large volumes of sequence data efficiently and reliably, we have developed the EASY system for performing combined protein sequence and pattern database searches. EASY runs searches simultaneously and distils results into a concise 1-line diagnosis. By bringing together results of several different analyses, EASY provides a rapid means of evaluating biological significance, minimising the risk of inferring false relationships, for example from relying exclusively on top BLAST hits. The program has been tested using a variety of protein families and was instrumental in resolving family assignments in a major update of the PRINTS database. (+info)
Clinicians' response to computerized detection of infections.
(24/376)
OBJECTIVE: To analyze whether computer-generated reminders about infections could influence clinicians' practice patterns and consequently improve the detection and management of nosocomial infections. DESIGN: The conclusions produced by an expert system developed to detect and manage infections were presented to the attending clinicians in a pediatric hospital to determine whether this information could improve detection and management. Clinician interventions were compared before and after the implementation of the system. MEASUREMENTS: The responses of the clinicians (staff physicians, physician assistants, and nurse practitioners) to the reminders were determined by review of paper medical charts. Main outcome measures were the number of suggestions to treat and manage infections that were followed before and after the implementation of COMPISS (Computerized Pediatric Infection Surveillance System). The clinicians' opinions about the system were assessed by means of a paper questionnaire distributed following the experiment. RESULTS: The results failed to show a statistical difference between the clinicians' treatment strategies before and after implementation of the system (P: > 0.33 for clinicians working in the emergency room and P: > 0.45 for clinicians working in the pediatric intensive care unit). The questionnaire results showed that the respondents appreciated the information presented by the system. CONCLUSION: The computer-generated reminders about infections were unable to influence the practice patterns of clinicians. The methodologic problems that may have contributed to this negative result are discussed. (+info)