• Whether reproducibility can be accurately estimated from published information has major implications not only for choosing what to believe or what is worth replicating, but also for how we assess and fund science. (bmj.com)
  • We performed two multi-centre reproducibility studies on standardised functional WC evaluation (RELY 1 and 2). (biomedcentral.com)
  • The Brazilian Reproducibility Initiative , which aims to replicate between 60 and a 100 experiments from Brazilian biomedical science using common laboratory methods, provides a unique opportunity to study this question. (bmj.com)
  • The RELY studies aimed to establish the inter-rater reproducibility (reliability and agreement) of 'functional evaluations' in patients with mental disorders applying for disability benefits and to compare the effect of limited versus intensive expert training on reproducibility. (biomedcentral.com)
  • Awareness of reproducibility issues in various areas of science has been on the rise in recent years, with systematic replication efforts in areas such as psychology , economics , cancer biology and social sciences arising in recent years. (bmj.com)
  • The question of whether researchers can estimate the reproducibility of published findings has been studied in replication initiatives in psychology (see also this ), economics and social sciences , and the answer is that they are reasonably good at it. (bmj.com)
  • Can we predict the reproducibility of biomedical studies? (bmj.com)
  • It was even demonstrated that laypeople without a PhD in psychology show some accuracy in predicting reproducibility in these studies. (bmj.com)
  • These include indicators of methodological rigor such as randomisation and blinding, which are widely thought to reduce risk of bias , as well as effect sizes and statistical results - which have been shown to correlate with reproducibility in psychology studies . (bmj.com)
  • In 2013, on the heels of several retraction scandals and studies showing reproducibility rates as low as 10 percent for peer-reviewed articles, the prominent scientific journal Nature dedicated a special issue to the concerns over irreproducibility. (duke.edu)
  • With this in mind, we are recruiting experimental researchers to predict the probability that the results will be replicated - as well as to explain the reasons for their predictions! (bmj.com)
  • Researchers from Smith College, Duke University and Amherst College looked at how introductory statistics students responded to a curriculum modified to stress reproducibility. (duke.edu)
  • The authors of the present study thus looked to their own classrooms for ways to incorporate the idea of reproducibility. (duke.edu)
  • Olavo Amaral is an associate professor at the Institute of Medical Biochemistry Leopoldo de Meis, at the Federal University of Rio de Janeiro and project coordinator of the Brazilian Reproducibility Initiative. (bmj.com)
  • CONCLUSIONS: Expert radiologists achieved substantial reproducibility for the PRECISE scoring system, especially when data were pooled together according to the absence/presence of radiological progression (PRECISE 1-3 vs 4-5). (ucl.ac.uk)
  • CONCLUSIONS Scanning laser Doppler flowmetry has an acceptable reproducibility and is appropriate for description of the effect of graded changes in P o 2 on retinal haemodynamics. (bmj.com)
  • The results of the test, published in Nature today (May 20), show that each team performed the analysis in a subtly different manner and that their conclusions varied as a result. (the-scientist.com)
  • In conclusion, the physical performance and especially the high intensities activities of elite soccer players present good reproducibility when different types of SSG were applied in 2 different sessions in the same period, even if the rules and formats were being manipulated allowing coaches to use it with a target time-motion characteristic and an equivalent workload week after week. (omicsonline.org)
  • Pathologists reported good reproducibility at both 1% and 50% cut points. (aacrjournals.org)
  • This study supports that the FFQ has a good reproducibility for nutrient and food intake, and can provide a valid estimate of several important nutrients during pregnancy. (biomedcentral.com)
  • The third session of the workshop consisted of three panels discussing how to move forward using statistics to improve reproducibility. (nap.edu)
  • At the conclusion, a written report will be issued to Congress that includes an assessment of current activities to improve reproducibility and replication highlighting examples of good practices and examine factors that adversely affect reproducibility and replication. (the-scientist.com)
  • The authors emphasized that the findings reflect a problem that affects all of science and not just psychology, and that there is room to improve reproducibility in psychology. (wikipedia.org)
  • The CTs increased slightly when measured twice over the 15-minute intervals (short-term reproducibility) but remained stable over a period of 2 weeks (long-term reproducibility). (arvojournals.org)
  • The new "Reproducibility of Research Results" module discusses factors that contribute to the lack of reproducibility and the resulting problems that can emerge. (citiprogram.org)
  • Much has been written about Fleischmann and Pons, but I have not found if their problem was lack of reproducibility (so they found a plot that looked good and just focused on that) or if there was something darker at work. (wordpress.com)
  • Intraclass correlation coefficients (ICC) were calculated using a random effects one way analysis of variance as a measure of reproducibility. (bmj.com)
  • Means and standard deviations of all flow results of a given subject were calculated, at t0-t6 and the coefficients of variation as a measure of reproducibility. (bmj.com)
  • Two methods of image processing, one using a baseline image and another using the individual scans, were used to compare the reproducibility of three summary measures, average retardation, integral, and retardation ratio. (nih.gov)
  • The purpose of this study was to quantify functional MR imaging and magnetoencephalographic measures of the extent of cortical activity and to compare the reproducibility of these measures and their sensitivity to incremental stimulus change. (ajnr.org)
  • Evaluating microkeratome efficacy by 3D corneal lamellar flap thickness accuracy and reproducibility using Artemis VHF digital ultrasound arc-scann. (nih.gov)
  • Descriptive statistics revealed central flap thickness accuracy and reproducibility. (nih.gov)
  • A systematic qualitative and quantitative statistical evaluation of microkeratome accuracy and reproducibility can be performed by measuring corneal flap thickness with arc-scanning VHF digital ultrasound. (nih.gov)
  • In spite of its potential usefulness, the accuracy and reproducibility of the gene expression measures obtained in this way are still open to doubt. (biomedcentral.com)
  • To determine the reproducibility of 3D proton magnetic resonance spectroscopic imaging ( 1 H-MRSI) of the human prostate in a multicenter setting at 1.5T. (wiley.com)
  • As we have sought to develop FLT for imaging tumor response, it is also necessary to determine the best way to measure FLT retention, including SUV and kinetic variables, and to determine the reproducibility in several tumor types. (aacrjournals.org)
  • To assess the repeatability and reproducibility of dynamic corneal response parameters measured by the Corvis ST (Oculus, Wetzlar, Germany). (hindawi.com)
  • Both Scheimpflug-Placido systems show high repeatability and reproducibility for posterior corneal surface measurement. (healio.com)
  • Our purpose was to establish objective criteria for measuring hip abduction angles on MRI after hip reduction and spica casting in patients with DDH, and evaluate reproducibility and reliability of angle measurement using these criteria. (springermedizin.de)
  • The high ICC and low standard deviation of independently evaluated hip abduction angles indicates high reproducibility of measurement. (springermedizin.de)
  • The goal of this study was to establish the clinical reproducibility of ceMRI for the measurement of infarct size. (ahajournals.org)
  • Reproducibility - Measurement variation with either multiple operators on multiple instruments on the same sample (but possibly multiple lots) in multiple locations. (horiba.com)
  • Infarct size measurement by contrast enhanced magnetic resonance can be used as a surrogate parameter for acute myocardial infarction treatment trials, if reproducibility is excellent. (onlinejacc.org)
  • Patients underwent infarct size measurement on 2 consecutive days for reproducibility assessment. (onlinejacc.org)
  • B. Thompson , " Exploring the Replicability of a Study's Results: Bootstrap Statistics for the Multivariate Case " , Educational and Psychological Measurement , vol. 55, no. 1, pp. 84-94, 1995. (iit.edu)
  • Central reproducibility (standard deviation of the mean) was 30.3 microm, with a range of 106 to 228 microm. (nih.gov)
  • To assess intraoperator repeatability and interoperator reproducibility, within-subject standard deviation (Sw) and test-retest repeatability (TRT, 2.77 Sw) were calculated. (healio.com)
  • Mainz, Germany, May 15, 2017 / B3C newswire / -- BioNTech Diagnostics GmbH today announced the publication of an international prospective multicenter study that demonstrates the high reproducibility of the MammaTyper® in vitro diagnostic test. (b3cnewswire.com)
  • OBJECTIVES: We aimed to determine the interobserver reproducibility of the Prostate Cancer Radiological Estimation of Change in Sequential Evaluation (PRECISE) criteria for magnetic resonance imaging in patients on active surveillance (AS) for prostate cancer (PCa) at two different academic centres. (ucl.ac.uk)
  • For interobserver reproducibility, OPA was 84.2% (95% CI, 82.8-85.5) for the 1% cut point and 81.9% (95% CI, 80.4-83.3) for the 50% cut point, and Cohen's κ coefficients were 0.68 (95% CI, 0.65-0.71) and 0.58 (95% CI, 0.55-0.62), respectively. (aacrjournals.org)
  • The training was found to have no or very little impact on intra- or interobserver reproducibility. (aacrjournals.org)
  • A multicenter study was conducted to assess the interlaboratory reproducibility of broth microdilution testing of the more common rapidly growing pathogenic mycobacteria. (asm.org)
  • Currently, however, no standardized testing method for this group of organisms exists, nor has the interlaboratory reproducibility of any method been assessed. (asm.org)
  • Twenty well-characterized isolates of methicillin-resistant Staphylococcus aureus were used to study the optimal resolution and interlaboratory reproducibility of pulsed-field gel electrophoresis (PFGE) of DNA macrorestriction fragments. (asm.org)
  • Although, the reproducibility of a 6 minute walk has been addressed in terms of a hospital test, simulated home test and actual home test (Guyatt), its inter-variability is not well studied. (clinicaltrials.gov)
  • Schonberg was therefore a little surprised to see the NARPS results, admitting, "it wasn't easy seeing this variability, but it was what it was. (the-scientist.com)
  • A sample of individuals aged above 20 years, of both genders, living in São Paulo, was used for the validation study (n = 77) and reproducibility study (n = 74) of the food frequency questionnaire. (scielosp.org)
  • The food frequency questionnaire analyzed has good validity and reproducibility for estimating the food consumption of adults in São Paulo compared to the reference method, so it is an appropriate instrument to be used in epidemiological studies on similar populations. (scielosp.org)
  • The aim of this study was to examine the reproducibility and validity (against biochemical biomarkers) of a semi-quantitative food frequency questionnaire (FFQ) in pregnant women. (biomedcentral.com)
  • Russell Poldrack, a cognitive neuroscientist at Stanford University and a strong proponent of reproducibility in science (the ability to replicate published studies and get the same results) is no stranger to testing the limits of commonly used neuroscience techniques (See Making the Connectome Personal: One Brain, Many Scans ). (dana.org)
  • We believe that the sharing of data is essential to scientific transparency and reproducibility, and the work of our center will extend the OpenfMRI project to encompass a complete online data analysis and sharing platform. (insidehpc.com)
  • His lab is also engaged in the development of neuroinformatics tools to help improve the reproducibility and transparency of neuroscience, including the OpenfMRI.org and Neurovault.org data sharing projects and the Cognitive Atlas ontology. (nsf.gov)
  • We tested the reproducibility of the assessment of PD-L1 expression in non-small cell lung cancer (NSCLC) tissue samples by pathologists. (aacrjournals.org)
  • Using this assay, we investigated the inter- and intraobserver reproducibility of pathologists' assessment of PD-L1 staining and the impact of training on reproducibility. (aacrjournals.org)
  • The presence of an identifiable effective regurgitant orifice improves reproducibility of VC and a central regurgitant jet predicts substantial agreement among multiple observers of PISA assessment. (onlinejacc.org)
  • For intraobserver reproducibility, the overall percent agreement (OPA) was 89.7% [95% confidence interval (CI), 85.7-92.6] for the 1% cut point and 91.3% (95% CI, 87.6-94.0) for the 50% cut point. (aacrjournals.org)
  • Reproducibility, interobserver, and intraobserver variabilities were assessed and compared by the Bland-Altman method. (onlinejacc.org)
  • In an ideal world, all research papers would be runnable: simply click to replicate all results, using the same setup as the authors. (psu.edu)
  • Harry seems to have had access to the data and code used and he couldn't replicate the results. (stodden.net)
  • Because the reproducibility of experimental results is an essential part of the scientific method, an inability to replicate the studies of others has potentially grave consequences for many fields of science in which significant theories are grounded on unreproducible experimental work. (wikipedia.org)
  • Reproducibility is the ability to produce a copy or duplicate, in this case it is the ability to replicate the results of the original studies. (wikipedia.org)
  • Reproducibility was assessed by the intraclass correlation coefficient. (asnjournals.org)
  • Voxels with approved spectral quality were included to calculate Bland-Altman parameters for reproducibility from the choline plus creatine to citrate ratio (CC/C). The repeated spectroscopic data were also evaluated with a standardized clinical scoring system. (wiley.com)
  • Nonetheless, collectively these results offer a clear conclusion: A large portion of replications produced weaker evidence for the original findings despite using materials provided by the original authors, review in advance for methodological fidelity, and high statistical power to detect the original effect sizes. (sciencemag.org)
  • Reproducibility can occur across different realms-numerical, empirical, computational and statistical-and may be analytical, direct, systematic or conceptual. (nsf.gov)
  • Priming may have affected our results, but the use of a combined task, in which naming is paired with noun generation, improves the reproducibility of activations and is also suitable for IOM. (ajnr.org)
  • The second panel on reporting scientific results and sharing scientific study data was moderated by Victoria Stodden (University of Illinois, Urbana-Champaign) and included the following panelists: Keith Baggerly (MD Anderson Cancer Center), Ronald Boisvert (Association for Computing Machinery and National Institute of Standards and Technology), Randy LeVeque (Society for Industrial and Applied Mathematics and University of Washington), and Marcia McNutt ( Science magazine). (nap.edu)
  • The Reproducibility Project: Cancer Biology launched in 2013 as an ambitious effort to scrutinize key findings in 50 cancer papers published in Nature, Science, Cell and other high-impact journals. (reproduciblescience.org)
  • The project pulled these studies from three different journals, Psychological Science, the Journal of Personality and Social Psychology, and the Journal of Experimental Psychology: Learning, Memory, and Cognition, published in 2008 to see if they could get the same results as the initial findings. (wikipedia.org)
  • Interpretation of experimental results cannot be performed without deep understanding of what happened to the data. (insidehpc.com)
  • and reproducibility of experimental results. (ouhsc.edu)
  • As a result, there is an increasingly urgent call for validation and verification of published research results, both within the academic community and the public at large (e.g. (springer.com)
  • Because scientific research represents a major public investment and is the basis for many decisions that we make in medicine and society, it is essential that we can trust the results. (insidehpc.com)
  • The frequent near-impossibility of verification of computational results when reproducibility is not considered a research goal is shown by the miserable travails of "Harry," a CRU employee with access to their system who was trying to reproduce the temperature results. (stodden.net)
  • CITI Program is pleased to announce the availability of a new "Reproducibility of Research Results" module. (citiprogram.org)
  • Reproducibility is a defining feature of science, but the extent to which it characterizes current research is unknown. (sciencemag.org)
  • In part two of our conversation with Chris Graf, Wiley's new Director of Research Integrity and Publishing Ethics, we discuss what reproducibility really means and share how the Committee on Publication Ethics (COPE) works to protect research integrity. (wiley.com)
  • From peer review to reproducibility, publishers, authors, and editors are all exploring how to be more transparent to improve research integrity and impact. (wiley.com)
  • Through this DCL, the participating programs noted above encourage proposals that specifically seek to reproduce, verify and/or characterize recent results in the disciplines covered under each program's ambit of research. (nsf.gov)
  • Proposers should be aware that the NSF merit review principles ( https://www.nsf.gov/pubs/policydocs/pappguide/nsf16001/gpg_3.jsp#IIIA1 ) as well as the NSF merit review criteria ( https://www.nsf.gov/pubs/policydocs/pappguide/nsf16001/gpg_3.jsp#IIIA2 ) support proposals that advance reproducibility in scientific research to the same extent as those that advance transformative research. (nsf.gov)
  • As the data collection ability of nearly every area of science has ballooned, so has the potential for problematic research practices that can lead to irreproducible results. (nsf.gov)
  • Computational neuroscience is a rapidly developing field, where reproducibility and comparability of research results have gained increasing interest over the past years. (frontiersin.org)
  • Reproducibility of research results is a founding principle of scientific methodology. (frontiersin.org)
  • In addition to advancing research and discovery in pure and applied mathematics, computation is pervasive across the sciences and now computational research results are more crucial than ever for public policy, risk management, and national security. (brown.edu)
  • Eand Medici Sciences , " Reproducibility Issues in Research with Animals and Animal Models " , National Academies Press, Washington, DC, 2015. (iit.edu)
  • We found reports documenting research on the kinetics of NNK formation in sidestream cigarette smoke by searching the ∼45 million pages of tobacco industry documents made public as a result of litigation against the tobacco companies. (aacrjournals.org)
  • The alarming claim is that many or even most biomedical research results are false. (neuron-eranet.eu)
  • Reproducibility of funded research projects is therefore one of the most important assets, since it clearly affects the perception and acceptance of research funding in society. (neuron-eranet.eu)
  • b) The funding organizations perspective was explored by research councils such as the Netherlands Organization for Scientific Research (NWO) and the Medical Research Council (MRC, UK) who have taken a lead in operationalization of measures to improve the reproducibility of data. (neuron-eranet.eu)
  • The guidelines received consensus between scientists, statisti-cians, journal editors, and research funders and are used to ensure reproducibility of animal research and avoid unnecessary animal use. (neuron-eranet.eu)
  • To explore which measures NEURON can embrace to maintain and improve international quality standards, a workshop on 'Quality assurance and reproducibility of results' was organized on January 19th, 2017 in Oslo. (neuron-eranet.eu)
  • Reproducibility is a rarely discussed, but absolutely critical aspect of trial implementation and data management. (cytel.com)
  • The data shows that MammaTyper® provides virtually identical results - independent of the location, the time of day, the instruments used and the operator performing the test (1). (b3cnewswire.com)
  • It appears as though the leak came about through a long battle to get the CRU scientists to reveal the code and data associated with published results, and highlights a crack in the scientific method as practiced in computational science. (stodden.net)
  • The merging and preprocessing of data in preparation for modeling and estimation encompasses a potentially very large number of steps, and a change in any one could produce different results. (stodden.net)
  • Responding with raw data when questioned about computational results is merely a canard, not intended to seriously facilitate reproducibility. (stodden.net)
  • However, little is known about the reproducibility of imaging data in the language domain. (ajnr.org)
  • Before imaging data can be applied within the framework of neurosurgical planning (eg, for determining the extent of a resection), data on the reproducibility of the language paradigms should be available. (ajnr.org)
  • Previous studies however showed weak reproducibility of data. (bmj.com)
  • If your collected data cannot be automatically linked (probably via metadata) to the exact instrument configuration at the time of collection, then you do not really have a scientific result: you have something suggestive but not convincing. (wordpress.com)
  • Where practical, proposers should also propose models and openly accessible repositories for complete data sharing of all results from these experiments. (nsf.gov)
  • These include an integrated platform for the analysis and open sharing of neuroimaging data, frameworks for the description of data and metadata, and the use of software containers and virtual machines to enhance computational reproducibility. (nsf.gov)
  • The big question being, if you analyze data in different ways, would you get different results? (dana.org)
  • The reproducibility of the probe data obtained hybridizing deer, Old-World primates, and human RNA samples to Affymetrix human GeneChip ® U133 Plus 2.0 was compared. (biomedcentral.com)
  • However, the reproducibility and accuracy of these measures largely depends on the use of appropriated algorithms to derive the gene expression data from the probe data. (biomedcentral.com)
  • Five papers had reproducibility data for IGT or IFG, two of which where from the same population but sampled differently. (degruyter.com)
  • In reproducibility correlation coefficients ranged from 0.36 (polyunsaturated fat) to 0.69 (calcium), and weighted kappa exceeded 0.40 for 80% of the nutrients. (scielosp.org)
  • The results were subsequently analyzed for their reproducibility both between the institutes (inter-site) and within each institute (intra-site). (b3cnewswire.com)
  • Assessing the reproducibility of activations found with language paradigms in a repeated-measures design poses a special problem with regard to selection of stimuli. (ajnr.org)
  • The established immunohistochemistry (IHC) method has, however, been subject to review for some time - especially in terms of the reproducibility and comparability of Ki-67 results (2,3). (b3cnewswire.com)
  • The scientific community across all disciplines faces the same challenges of ensuring accessibility, reproducibility, and efficient comparability of scientific results. (frontiersin.org)
  • Dellal A, Chamari K, Payet F, Djaoui L, Wong DP (2016) Reproducibility of Physical Performance during Small- and Large-sided Games in Elite Soccer in Short Period: Practical Applications and Limits. (omicsonline.org)
  • NATURE, 452, Vol. 533, May 2016) that had asked scientists what led to problems in reproducibility. (neuron-eranet.eu)
  • There is concern about the rate and predictors of reproducibility, but limited evidence. (sciencemag.org)
  • Title: Inter-observer and intra-observer reproducibility in the histo-pathology of follicular thyroid carcinoma. (thyroidmanager.org)
  • Fewer than half of the attempted replications were successful at producing statistically significant results in the expected directions, though most of the attempted replications did produce trends in the expected directions. (wikipedia.org)
  • In their initial publications 97 of these 100 studies claimed to have significant results. (wikipedia.org)
  • If NNK formation also occurs in the lower smoke concentrations observed in real smoking environments, these results suggest that nitrosation of nicotine and/or nicotine breakdown products in aging secondhand smoke is a significant contributor to nitrosamine exposure in humans. (aacrjournals.org)