Manuscript peer review: a helpful checklist for students and novice referees.
(1/41)
The ability to contribute consistent, fundamentally sound critiques is an essential element of the scientific peer review process and an important professional skill for investigators. Despite its importance, many students and junior scientists do not have an adequate working knowledge of how to effectively critique research manuscripts. Part of the problem, in our view, is that novice referees often lack a comprehensive understanding of the basic issues that should be considered in evaluating scientific articles. Specifically, they tend to overemphasize certain limitations (usually methodological), while missing other key points related to the scientific method that should be weighed much more heavily. In our journal club and graduate courses we have been using a "checklist" to help graduate students and postdoctoral fellows critically analyze original research papers. In this article we present these guidelines in the hope that they will serve as a helpful resource for students and other novice reviewers when critiquing scientific manuscripts. (+info)
Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone?
(2/41)
We aimed to determine the reproducibility of assessments made by independent reviewers of papers submitted for publication to clinical neuroscience journals and abstracts submitted for presentation at clinical neuroscience conferences. We studied two journals in which manuscripts were routinely assessed by two reviewers, and two conferences in which abstracts were routinely scored by multiple reviewers. Agreement between the reviewers as to whether manuscripts should be accepted, revised or rejected was not significantly greater than that expected by chance [kappa = 0.08, 95% confidence interval (CI) -0.04 to -0.20] for 179 consecutive papers submitted to Journal A, and was poor (kappa = 0.28, 0.12 to 0. 40) for 116 papers submitted to Journal B. However, editors were very much more likely to publish papers when both reviewers recommended acceptance than when they disagreed or recommended rejection (Journal A, odds ratio = 73, 95% CI = 27 to 200; Journal B, 51, 17 to 155). There was little or no agreement between the reviewers as to the priority (low, medium, or high) for publication (Journal A, kappa = -0.12, 95% CI -0.30 to -0.11; Journal B, kappa = 0.27, 0.01 to 0.53). Abstracts submitted for presentation at the conferences were given a score of 1 (poor) to 6 (excellent) by multiple independent reviewers. For each conference, analysis of variance of the scores given to abstracts revealed that differences between individual abstracts accounted for only 10-20% of the total variance of the scores. Thus, although recommendations made by reviewers have considerable influence on the fate of both papers submitted to journals and abstracts submitted to conferences, agreement between reviewers in clinical neuroscience was little greater than would be expected by chance alone. (+info)
Report of the world association of medical editors: agenda for the future.
(3/41)
During a 3-day meeting at Bellagio in January 2001, a group of 20 editors from 12 countries in 5 continents met to map out a strategy for the World Association of Medical Editors (WAME)'s continued development in the service of medical editors over the next several years. The group: 1) Developed a statement of principles on the standards of professionalism and responsibilities of editors (this statement will be posted on the Web site after electronic consultation with and comment by WAME editors); 2) Agreed to assess the extent to which these principles are reflected in practice and to explore barriers to their adoption, using data from a survey and focus groups; 3) Developed and outlined an on-line program for distance learning, targeted at new editors; 4) Planned for formal evaluation of the educational outreach program; and 5) Agreed to support regional initiatives to strengthen local editorial capacity. Underpinning all past and proposed future activities is the WAME Web site. The ambitious plans outlined above will require extensive development of the site, plans for which were made at the Bellagio meeting. (+info)
Digital plagiarism--the Web giveth and the Web shall taketh.
(4/41)
Publishing students' and researchers' papers on the World Wide Web (WWW) facilitates the sharing of information within and between academic communities. However, the ease of copying and transporting digital information leaves these authors' ideas open to plagiarism. Using tools such as the Plagiarism.org database, which compares submissions to reports and papers available on the Internet, could discover instances of plagiarism, revolutionize the peer review process, and raise the quality of published research everywhere. (+info)
Language and publication in "Cardiovascular Research" articles.
(5/41)
BACKGROUND: The acceptance rate of non-mother English tongue authors is generally a lot lower than for native English tongue authors. Obviously the scientific quality of an article is the principal reason for publication. However, is editorial rejection purely on scientific grounds? English mother tongue writers publish more than non mother-tongue writers--so are editors discriminating linguistically? We therefore decided to survey language errors in manuscripts submitted for publication to Cardiovascular Research (CVR). METHOD: We surveyed language errors in 120 medical articles which had been submitted for publication in 1999 and 2000. The language "error" categories were divided into three principal groups: grammatical, structural and lexical which were then further sub-divided into key areas. The articles were corrected without any knowledge of the author's nationality or the corrections made by other language researchers. After an initial correction, a sample of the papers were cross-checked to verify reliability. RESULTS: The control groups of US and UK authors had an almost identical acceptance rate and overall "error" rate indicating that the language categories were objective categories also for the other nationalities. Although there was not a direct relationship between the acceptance rate and the amount of language errors, there was a clear indication that badly written articles correlated with a high rejection rate. The US/UK acceptance rate of 30.4% was higher than for all the other countries. The lowest acceptance rate of 9% (Italian) also had the highest error rate. DISCUSSION: Many factors could influence the rejection of an article. However, we found clear indications that carelessly written articles could often have either a direct or subliminal influence on whether a paper was accepted or rejected. On equal scientific merit, a badly written article will have less chance of being accepted. This is even if the editor involved in rejecting a paper does not necessarily identify language problems as a motive for rejection. A more detailed look at the types and categories of language errors is needed. Furthermore we suggest the introduction of standardised guidelines in scientific writing. (+info)
Development and evaluation of a quality score for abstracts.
(6/41)
BACKGROUND: The evaluation of abstracts for scientific meetings has been shown to suffer from poor inter observer reliability. A measure was developed to assess the formal quality of abstract submissions in a standardized way. METHODS: Item selection was based on scoring systems for full reports, taking into account published guidelines for structured abstracts. Interrater agreement was examined using a random sample of submissions to the American Gastroenterological Association, stratified for research type (n = 100, 1992-1995). For construct validity, the association of formal quality with acceptance for presentation was examined. A questionnaire to expert reviewers evaluated sensibility items, such as ease of use and comprehensiveness. RESULTS: The index comprised 19 items. The summary quality scores showed good interrater agreement (intra class coefficient 0.60 - 0.81). Good abstract quality was associated with abstract acceptance for presentation at the meeting. The instrument was found to be acceptable by expert reviewers. CONCLUSION: A quality index was developed for the evaluation of scientific meeting abstracts which was shown to be reliable, valid and useful. (+info)
SOURCE MATERIALS AND THE LIBRARY: THE DISPERSION OF THE BEAUMONT PAPERS.
(7/41)
The author discusses the confusion, attributable to misleading references, which long obscured the location of the main portion of the William Beaumont papers. The collection was previously thought to be in the National Library of Medicine, but it is actually housed in the Washington University School of Medicine Library. This has been clarified by the discovery of a series of previously unpublished letters of Sir William Osler, as well as by other newly found materials. The history of the Washington University School of Medicine Library is reviewed, and the facts relating to the disposition of the Beaumont collection in that library rather than the National Library of Medicine are told. Also considered are some of the problems which the dispersion of documentary source materials creates for the scholar. With the Beaumont papers as a case in point, various locations of such materials are cited, and their contents briefly noted. (+info)
Effects of a reviewer-prompting strategy on timely manuscript reviews.
(8/41)
We studied a reviewer-prompting system designed to improve the timeliness of journal reviews. The prompting system consisted of an e-mail message sent individually to reviewers noting the manuscript number, review due date, and associated social amenities for the timely completion of the task. Our results indicated that the prompting system increased timely reviews. (+info)