Using a peer evaluation system to assess faculty performance and competence.
(57/287)
BACKGROUND AND OBJECTIVES: Identification of reliable methods to evaluate the newly mandated American Board of Medical Specialties (ABMS)/Accreditation Council for Graduate Medical Education (ACGME) competencies of the board-certified physician is in its early stages. In this study, we evaluated a comprehensive faculty peer evaluation system designed to assess the six competencies as well as faculty performance in their primary departmental roles and teaching. METHODS: Using a one-page form containing 19 items, all faculty members evaluated all other faculty within a single department. Annual individual faculty reviews included discussion of these aggregated evaluations. RESULTS: The reliabilities for the ACGME competency subscales ranged from .61 to .79. While overall scores were relatively high, there was variability across faculty. Factor analysis demonstrated that evaluation items load onto three scales. The first relates to clinical practice and teaching, the second to departmental citizenship, and the third to research. An item related to systems-based practice loaded on none of the factors. Research faculty outscored other faculty on the items reflecting research skills. Faculty who had primary administrative responsibility scored higher than other faculty on measures related to role within the department. No differences in subgroup scores for clinical skills were observed. CONCLUSIONS: Using a method in which all faculty evaluate each other can result in objective, reliable measures of faculty performance. (+info)
"Is Cybermedicine Killing You?"--The story of a Cochrane disaster.
(58/287)
This editorial briefly reviews the series of unfortunate events that led to the publication, dissemination, and eventual retraction of a flawed Cochrane systematic review on interactive health communication applications (IHCAs), which was widely reported in the media with headlines such as "Internet Makes Us Sick," "Knowledge May Be Hazardous to Web Consumers' Health," "Too Much Advice Can Be Bad for Your Health," "Click to Get Sick?," and even "Is Cybermedicine Killing You?". While the media attention helped to speed up the identification of errors, leading to a retraction of the review after only 13 days, a paper published in this issue of JMIR by Rada shows that the retraction, in contrast to the original review, remained largely unnoticed by the public. We discuss the three flaws of the review, which include (1) data extraction and coding errors, (2) the pooling of heterogeneous studies, and (3) a problematic and ambiguous scope and, possibly, some overlooked studies. We then discuss "retraction ethics" for researchers, editors/publishers, and journalists. Researchers and editors should, in the case of retractions, match the aggressiveness of the original dissemination campaign if errors are detected. It is argued that researchers and their organizations may have an ethical obligation to track down journalists who reported stories on the basis of a flawed study and to specifically ask them to publish an article indicating the error. Journalists should respond to errors or retractions with reports that have the same prominence as the original story. Finally, we look at some of the lessons for the Cochrane Collaboration, which include (1) improving the peer-review system by routinely sending out pre-prints to authors of the original studies, (2) avoiding downplay of the magnitude of errors if they occur, (3) addressing the usability issues of RevMan, and (4) making critical articles such as retraction notices open access. (+info)
GP experiences of partner and external peer appraisal: a qualitative study.
(59/287)
BACKGROUND: Appraisal is being adopted both in the UK and internationally as a means of aiding personal development for family doctors. However, it is not clear by whom they should be appraised. AIM: To explore attitudes of GPs towards being appraised by externally appointed GP colleagues and by their own partners. DESIGN OF STUDY: Semi-structured interviews of GPs who had experienced both forms of appraisal. SETTING: Lothian, Scotland. METHOD: Sixty-six GPs agreed to take part in a study of partner (n = 46) and external (n = 20) peer-based appraisal. Six months later this group was followed up by questionnaire to determine views of the process, in order to obtain a purposeful sample of 13 GPs who were interviewed in depth. RESULTS: We uncovered concern and a need for clarity about the linkage of appraisal to revalidation. Interviewees felt that the potentially charged nature of appraisal could lead to collusion between appraiser and appraisee, which may lead to a superficial engagement. Similarly, lack of local knowledge of an appraisee potentially enabled a strategy of avoidance. GPs opting for partner appraisal were less likely to undergo appraisal due to lack of protected time. CONCLUSION: There are reported advantages and disadvantages to having an external peer or partner appraisal. The relationship between revalidation and appraisal needs to be clarified as this leads to collusion and avoidance strategies by both appraisers and appraisees. Good training is required to both recognise and address these strategies. Protected time is essential for effective appraisal. (+info)
GP perceptions of appraisal: professional development, performance management, or both?
(60/287)
GPs' perceptions of the tension between the professional development and revalidation aspects of the current GP appraisal scheme were analysed. Evidence was gathered from focus groups representing general practice in Northern Ireland. The results indicate that there is support for the professional development aspects of appraisal but the link with revalidation is problematic, thereby potentially undermining GP support for the scheme. Greater clarity about the precise nature of the linkage is required to avoid a process that fails to fully satisfy the requirements of either appraisal or revalidation. (+info)
Can scientists and policy makers work together?
(61/287)
This paper addresses a fundamental question in evidence based policy making--can scientists and policy makers work together? It first provides a scenario outlining the different mentalities and imperatives of scientists and policy makers, and then discusses various issues and solutions relating to whether and how scientists and policy makers can work together. Scientists and policy makers have different goals, attitudes toward information, languages, perception of time, and career paths. Important issues affecting their working together include lack of mutual trust and respect, different views on the production and use of evidence, different accountabilities, and whether there should be a link between science and policy. The suggested solutions include providing new incentives to encourage scientists and policy makers to work together, using knowledge brokers (translational scientists), making organisational changes, defining research in a broader sense, re-defining the starting point for knowledge transfer, expanding the accountability horizon, and finally, acknowledging the complexity of policy making. It is hoped that further discussion and debate on the partnership idea, the need for incentives, recognising the incompatibility problems, the role of civil society, and other related themes will lead to new opportunities for further advancing evidence based policy and practice. (+info)
Using a 'peer assessment questionnaire' in primary medical care.
(62/287)
BACKGROUND: Periodic assessment of clinician performance or 'revalidation' is being actively considered to reassure the public that doctors are 'up to date and fit to practice'. There is, therefore, increasing interest in how to assess individual clinician performance in a valid and reliable way. The use of peer assessment questionnaires is one of the methods being considered and investigated by the General Medical Council in the UK. AIM: To test the feasibility of using a peer assessment questionnaire in a primary care setting, and consider the related issues of validity and reliability and compare the results to previous studies. DESIGN: Cross-sectional survey in a volunteer sample. SETTING: General practice in the UK. METHOD: GPs who volunteered to take part in an evaluation of a pilot appraisal implementation scheme were recruited by appraisers. These volunteers (GP subjects) chose 15 colleagues to complete a 'peer assessment' questionnaire that asked peers to make judgements about their clinical skills and other characteristics, such as 'compassion', 'integrity' and 'responsibility'. RESULTS: Of the 207 practitioners that agreed to be appraised, 113 completed the optional task of implementing the peer questionnaire. Of the 1271 raters, 1189 provided data about their roles and 33.6% of these were GPs. The data revealed significant levels of items where peers were 'unable to evaluate' the issues posed in the questionnaire (ranging from 13.7-1.8%). These rates differed from those obtained in studies based in the US where mean scores were slightly higher. Although the overall results are broadly similar to those previously obtained, there are sufficient differences to suggest that there are contextual issues influencing the interpretation of the items and therefore the scoring process. CONCLUSION: The volunteer sample in this study found no major obstacles to the implementation of the peer assessment questionnaire. While it is not possible to generalise from this selected volunteer sample, the use of peer assessment questionnaires appears feasible and may be acceptable to clinical practitioners. However, concern remains about the validity of such instruments and that their development did not fully consider issues of procedural justice or whether the overall purpose of the tools was to be formative, summative, or both. (+info)
What makes the best medical ethics journal? A North American perspective.
(63/287)
BACKGROUND: There currently exist no data on the factors that contribute to determining why medical ethicists choose to review for and submit articles to medical ethics journals. OBJECTIVE: To establish which factors contribute to medical ethicists reviewing articles for or submitting them to medical ethics journals by consulting those who are active in this capacity. METHODS: Medical ethicists were surveyed to determine their incentives and disincentives for reviewing articles for or submitting them to medical ethics journals. Survey participants were chosen based on a review of the academic and research record of medical ethicists working in North America in higher education institutions. RESULTS: The most frequent incentives to reviewing journal articles were: an opportunity to contribute to the field/profession, the good reputation of the journal, the high impact factor of the journal, and to keep up to date on current research. The most frequent disincentives to reviewing journal articles were: time constraints due to academic commitments, the poor reputation of the journal, and time constraints caused by other editorial commitments (for example, reviewing for other journals/publishers). The most important incentives to submitting journal articles were: the good reputation of the journal, the quality of scholarship previously published in the journal, the impact factor of the journal, and a fast turn-around from acceptance to publication. The most important disincentives to submitting journal articles were: the poor reputation of the journal, the poor quality of work previously published in the journal, and a slow turn-around from acceptance to publication. CONCLUSION: A series of factors that medical ethics journals should strive to employ to encourage reviewing and submission of articles are recommended. (+info)
Percutaneous coronary intervention: recommendations for good practice and training.
(64/287)
Cardiologists undertaking percutaneous coronary intervention (PCI) are excited by the combination of patient and physician satisfaction and technological advance occurring on the background of the necessary manual dexterity. Progress and applicability of percutaneous techniques since their inception in 1977 have been remarkable; a sound evidence base coupled with the enthusiasm and ingenuity of the medical device industry has resulted in a sea change in the treatment of coronary heart disease (CHD), which continues to evolve at breakneck speed. This is the third set of guidelines produced by the British Cardiovascular Intervention Society and the British Cardiac Society. Following the last set of guidelines published in 2000, we have seen PCI activity in the UK increase from 33,652 to 62,780 (87% in four years) such that the PCI to coronary artery bypass grafting ratio has increased to 2.5:1. The impact of drug eluting stents has been profound, and the Department of Health is investigating the feasibility of primary PCI for acute myocardial infarction. Nevertheless, the changes in the structure of National Health Service funding are likely to focus our attention on cost effective treatments and will require physician engagement and sensitive handling if we are to continue the rapid and appropriate growth in our chosen field. It is important with this burgeoning development now occurring on a broad front (in both regional centres and district general hospitals) that we maintain our vigilance on audit and outcome measures so that standards are maintained for both operators and institutions alike. This set of guidelines includes new sections on training, informed consent, and a core evidence base, which we hope you will find useful and informative. (+info)