Tag Archives: open science

Fostering Reproducibility and Replicability in Science

Scientific research informs decisions that address many pressing issues, but what happens when results from one lab or study cannot be confirmed in another? Inconsistent results undermine the validity of scientific findings and contribute to the growing concern about replicability and reproducibility in science. A widespread strategy involving a variety of stakeholders is essential in order to promote openness and transparency in the research enterprise.

Three recent reports from the National Academies identify opportunities for meaningful improvement in research practices and offer guidance toward open, consistent, and objective science. Most recently, our 2019 report, Reproducibility and Replicability in Science, defines the terms “reproducibility” and “replicability” as distinct concepts that are each critical in achieving this goal. While many use these terms interchangeably, this differentiation is a critical step towards stronger scientific research practices and more reliable science. To learn more about open science, read or download our reports for free.

Reproducibility and Replicability in Science

One of the pathways by which the scientific community confirms the validity of a new scientific discovery is by repeating the research that produced it. When a scientific effort fails to independently confirm the computations or results of a previous study, some fear that it may be a symptom of a lack of rigor in science, while others argue …

[more]

Methods to Foster Transparency and Reproducibility of Federal Statistics: Proceedings of a Workshop

In 2014 the National Science Foundation (NSF) provided support to the National Academies of Sciences, Engineering, and Medicine for a series of Forums on Open Science in response to a government-wide directive to support increased public access to the results of research funded by the federal government. However, the breadth of the work …

[more]

Open Science by Design: Realizing a Vision for 21st Century Research

Openness and sharing of information are fundamental to the progress of science and to the effective functioning of the research enterprise. The advent of scientific journals in the 17th century helped power the Scientific Revolution by allowing researchers to communicate across time and space, using the technologies of that era to generate …

[more]

Fostering Integrity in Research

The integrity of knowledge that emerges from research is based on individual and collective adherence to core values of objectivity, honesty, openness, fairness, accountability, and stewardship. Integrity in science means that the organizations in which research is conducted encourage those involved to exemplify these values in every step of …

[more]

Statistical Challenges in Assessing and Fostering the Reproducibility of Scientific Results: Summary of a Workshop

Questions about the reproducibility of scientific research have been raised in numerous settings and have gained visibility through several high-profile journal and popular press articles. Quantitative issues contributing to reproducibility challenges have been considered (including improper data measurement and analysis, inadequate statistical …

[more]

Reproducibility Issues in Research with Animals and Animal Models: Workshop in Brief

In June 2014, the Roundtable on Science and Welfare in Laboratory Animal Use organized a public workshop to discuss fundamental aspects of experimental design of research using animals and animal models, aimed at improving reproducibility. At this workshop, researchers from around the world explored the many facets of animal-based research that …

[more]

Sharing Science to Promote Discovery

Last week, an international team of scientists identified an impact crater in Greenland using data from NASA’s IceBridge Project. Our recent report, Open Science by Design, explores opportunities and challenges in making research freely available and usable. It’s free to download, along with the other related titles below.

Open Science by Design: Realizing a Vision for 21st Century Research

[more]

The Future of Scientific Knowledge Discovery in Open Networked Environments: Summary of a Workshop

Digital technologies and networks are now part of everyday work in the sciences, and have enhanced access to and use of scientific data, information, and literature significantly. They offer the promise of accelerating the discovery and …

[more]

Sharing Clinical Research Data: Workshop Summary

Pharmaceutical companies, academic researchers, and government agencies such as the Food and Drug Administration and the National Institutes of Health all possess large quantities of clinical research data. If these data were shared more widely …

[more]

The Case for International Sharing of Scientific Data: A Focus on Developing Countries: Proceedings of a Symposium

The theme of this international symposium is the promotion of greater sharing of scientific data for the benefit of research and broader development, particularly in the developing world. This is an extraordinarily important topic. Indeed, I have …

[more]

Principles and Obstacles for Sharing Data from Environmental Health Research: Workshop Summary

On March 19, 2014, the National Academies of Sciences, Engineering, and Medicine held a workshop on the topic of the sharing of data from environmental health research. Experts in the field of environmental health agree that there are benefits to …

[more]

Discussion Framework for Clinical Trial Data Sharing: Guiding Principles, Elements, and Activities

Sharing data generated through the conduct of clinical trials offers the promise of placing evidence about the safety and efficacy of therapies and clinical interventions on a firmer basis and enhancing the benefits of clinical trials. …

[more]

Using Data Sharing to Improve Coordination in Peacebuilding: Report of a Workshop by the National Academy of Engineering and United States Institute of Peace: Roundtable on Technology, Science, and Peacebuilding

On May 23, 2012, the Roundtable on Technology, Science, and Peacebuilding convened a workshop at the United States Institute of Peace (USIP) to investigate data sharing as a means of improving coordination among US government and nongovernment …

[more]

Ensuring the Integrity, Accessibility, and Stewardship of Research Data in the Digital Age

As digital technologies are expanding the power and reach of research, they are also raising complex issues. These include complications in ensuring the validity of research data; standards that do not keep pace with the high rate of innovation; …

[more]

The Dual-Use Dilemma: Open Science vs. National Security

In late March, the National Institutes of Health announced the U.S. Government Policy for Oversight of Life Sciences Dual Use Research of Concern, which establishes regular review of federally-funded or conducted research with certain high-consequence pathogens and toxins with potential for dual use. This review is the latest in an ongoing effort to preserve the benefits of life sciences research and open scientific communication while minimizing the risk of misuse by those who wish to do harm.

Long before the issue of dual-use biological research rose to its current prominence, the National Research Council (NRC) led the debate on the tension between open scientific communication and national security. In 1982 the NRC produced the landmark report Scientific Communication and National Security. The message of this report remains relevant today.

Scientific Communication and National Security addresses one of the most difficult of policy issues: one in which fundamental national objectives seem to have been abruptly thrown into direct conflict. Advances in science and technology have traditionally thrived in an atmosphere of open communication; openness has contributed to American military and economic strength and has been a tenet of American culture and higher education. However, recent trends, including apparent increases in acquisition efforts by our adversaries, have raised serious concerns that openness may harm U.S. security by providing adversaries with militarily relevant technologies that can be directed against us. As would be expected when major national interests are in question, signs of distrust have appeared on all sides of the growing public discussion. The federal government, through its research and development agencies, and the university research community, where most basic research is conducted, both will lose much if the nation cannot find a policy course that reflects legitimate concerns.

This report, as well as others on the subject of dual use, is available to read or download at no charge.

Reports

Scientific Communication and National Security204 pages | Paperback | Price: $50.62The military, political, and economic preeminence of the United States during the post-World War II era is based to a substantial degree on its superior rate of achievement in science and technology, as well as on its capacity to translate these achievements… [more]

Beyond ‘Fortress America’: National Security Controls on Science and Technology in a Globalized World150 pages | Paperback | Price: $37.80The national security controls that regulate access to and export of science and technology are broken. As currently structured, many of these controls undermine our national and homeland security and stifle American engagement in the global economy, and in… [more]

Science and Security in a Post 9/11 World: A Report Based on Regional Discussions Between the Science and Security Communities138 pages | Paperback | Price: $27.67Based on a series of regional meetings on university campuses with officials from the national security community and academic research institutions, this report identifies specific actions that should be taken to maintain a thriving scientific research… [more]

Biotechnology Research in an Age of Terrorism164 pages | Paperback | Price: $31.50In recent years much has happened to justify an examination of biological research in light of national security concerns. The destructive application of biotechnology research includes activities such as spreading common pathogens or transforming them into… [more]

Seeking Security: Pathogens, Open Access, and Genome Databases88 pages | Paperback | Price: $18.90Within the last 30 years, the genomes of thousands of organisms, from viruses, to bacteria, to humans, have been sequenced or partially sequenced and deposited in databases freely accessible to scientists around the world. This information is accelerating… [more]

Research in the Life Sciences with Dual Use Potential: An International Faculty Development Project on Education About the Responsible Conduct of Science56 pages | Paperback | Price: $26.10In many countries, colleges and universities are where the majority of innovative research is done; in all cases, they are where future scientists receive both their initial training and their initial introduction to the norms of scientific conduct regardless… [more]

Challenges and Opportunities for Education About Dual Use Issues in the Life Sciences146 pages | Paperback | Price: $22.50The Challenges and Opportunities for Education About Dual Use Issues in the Life Sciences workshop was held to engage the life sciences community on the particular security issues related to research with dual use potential. More than 60 participants from… [more]