Computational Notebooks for Reproducible Research: Best Practices

Computational notebooks such as Jupyter and R Markdown combine executable code, descriptive text, and output in a single interactive file, making them a practical mechanism for documenting and sharing research workflows. This article provides a structured approach for students, researchers, and life-science professionals who want to use notebooks to improve the reproducibility of their work. The guidance covers version control, environment management, sharing practices, and a checklist for creating reproducible notebooks.

At a Glance

Computational notebooks serve as digital documentation that contains descriptive text alongside code and its outputs in one dynamic file that nonspecialists can understand more easily than raw code alone. The table below summarizes the core practices covered in this article and the problems they address.

Why Computational Notebooks Matter for Reproducible Research

Reproducibility of computational aspects is a key component of scientific reproducibility. Jupyter notebooks facilitate the bundling of executable code with its documentation and output in one interactive environment, and they represent a popular mechanism to document and share computational workflows, including for research publications. A large-scale analysis of Jupyter notebooks associated with biomedical publications found that out of 27,271 notebooks from 2,660 GitHub repositories linked to 3,467 publications, only 1,203 notebooks ran through without any errors when dependencies could be installed. This finding demonstrates that creating a notebook is not sufficient for reproducibility, and deliberate practices are required.

The ubiquitous use of computational work for data generation, processing, and modeling has increased the importance of digital documentation in improving research quality and impact. Computational notebooks are files that contain descriptive text, as well as code and its outputs, in a single dynamic file that is easier to understand by nonspecialists. Traditionally used by data scientists when producing reports and informing decision-making, the use of this tool in research publication is not common, despite its potential to increase research impact and quality.

For a single study, the content of notebook documentation partially overlaps with that of classical lab notebooks and the scientific manuscript reporting the study. A starter kit approach can facilitate the implementation of computational notebooks in the research process, including publication. The kit contains the template of a computational notebook integrated into a research project that employs R, Python, or Julia. Using examples of ecological studies, such kits show how computational notebooks foster the implementation of principles of open science, such as reproducibility and traceability.

Core Principles of Notebook-Based Reproducibility

Predictability, Computability, and Stability

The predictability, computability, and stability framework for veridical data science provides a workflow and documentation approach that aims to produce responsible, reliable, reproducible, and transparent results across the data science life cycle. The workflow uses predictability as a reality check and considers the importance of computation in data collection, storage, and algorithm design. It augments predictability and computability with an overarching stability principle that expands on statistical uncertainty considerations to assess how human judgment calls impact data results through data and model or algorithm perturbations.

The framework proposes documentation based on R Markdown or Jupyter Notebook, with publicly available reproducible codes and narratives to back up human choices made throughout an analysis. This approach is demonstrated in a genomics case study available on Zenodo.

Dynamic Report Generation

A reproducible data analysis workflow combines the benefits of various open-source software tools including R Markdown, Git, Make, and Docker. The interplay of these tools ensures seamless integration of version management, dynamic report generation conforming to various journal styles, and full cross-platform and long-term computational reproducibility. The workflow ensures that the reporting of statistical results is consistent with the actual statistical results through dynamic report generation, the analysis exactly reproduces at a later point in time even if the computing platform or software is changed, and changes at any time during development and post-publication are tracked, tagged, and documented while earlier versions of both data and code remain accessible.

Dynamic document generation and version management alone are not sufficient to ensure long-term computational reproducibility. Combining containerization, dependence management, version management, and dynamic document generation increases scientific productivity by facilitating later reproducibility and reuse of code and data.

Documentation as Part of the Research Process

Research documentation is a central topic in reproducible computational research. Electronic laboratory notebooks and Jupyter notebooks are among the documentation approaches used in computational disciplines. It has become common practice to share data and programming codes used for numerical calculations to facilitate reproducibility and to foster collaborations that drive projects further by introducing new ideas, growing the data, and augmenting the code.

The reproducibility of experiments has been a long-standing impediment for further scientific progress. Computational methods have been instrumental in drug discovery efforts owing to their multifaceted utilization for data collection, pre-processing, analysis, and inference. In computational disciplines, sharing data and code has become common practice to facilitate reproducibility and foster collaboration.

Setting Up a Reproducible Notebook Workflow

Choose Your Notebook System

Jupyter and R Markdown are the two most widely used notebook systems in research. Jupyter supports multiple programming languages including Python, R, and Julia. R Markdown is integrated into the R ecosystem and supports dynamic report generation. Both systems allow you to combine narrative text, code, and output in a single document.

The GenePattern Notebook environment provides an example of integrating notebook capabilities with an investigator-focused interface that provides access to hundreds of genomic tools without the need to write code. This demonstrates that notebook systems can be adapted for users with varying programming experience.

For economic modeling, DynareR harnesses Dynare for economic modeling in R, R Markdown, and Quarto. For clinical trial reporting, R Markdown and R Shiny can be integrated to create dynamic reproducible reports that incorporate data analysis, tables, figures, and narrative content, ensuring real-time updates with the latest data.

Structure Your Notebook Project

A reproducible notebook project should separate the following components:

1. Raw data files that are never modified
2. Processed data files generated by analysis scripts
3. Notebook files containing narrative and code
4. Dependency files listing required software packages
5. Documentation files explaining the project structure
6. Output files such as figures and tables

The DataDesc metadata schema provides an approach for annotating data models in software interfaces and datasets. The ioProc workflow manager bundles shared transformation functions into adapter workflows and ensures transparent process documentation. This framework bridges the gap between abstract guidelines such as the FAIR principles and daily data and software driven analyses.

Manage Dependencies Explicitly

Dependency management is one of the most critical aspects of notebook reproducibility. A large-scale study found that out of 22,578 Python notebooks with dependencies declared in standard requirement files, dependencies could be installed successfully for 15,817 notebooks. This means that a substantial fraction of notebooks had dependency issues that prevented installation.

For Python notebooks, declare dependencies in a requirements.txt file or use an environment manager such as conda. For R Markdown, record the R version and package versions using the sessionInfo function or the renv package. Record the exact versions of all packages used in the analysis.

Use Version Control from the Start

Version control with Git allows you to track changes to code and narrative over time. Initialize a Git repository at the beginning of your project and commit changes regularly with descriptive messages. Tag releases or publication versions so that earlier versions of both data and code remain accessible.

The workflow combining R Markdown, Git, Make, and Docker ensures that changes at any time during development and post-publication are tracked, tagged, and documented while earlier versions of both data and code remain accessible. This approach is essential for long-term reproducibility.

Containerize for Long-Term Reproducibility

Containerization with Docker preserves the full computing environment including the operating system, software versions, and dependencies. This ensures that the analysis reproduces at a later point in time even if the computing platform or software has changed. Containers are particularly important for long-term reproducibility because software updates can break analyses even when dependency files are preserved.

The combination of containerization, dependence management, version management, and dynamic document generation increases scientific productivity by facilitating later reproducibility and reuse of code and data.

Practical Implementation Steps

Step 1: Create a Project Directory Structure

Create a directory structure that separates raw data, processed data, code, notebooks, and outputs. Use clear naming conventions that indicate the content and version of each file. Store raw data in a read-only location and never modify it directly.

Step 2: Write the Notebook with Reproducibility in Mind

Write the notebook as a narrative that explains the research question, the data, the analysis methods, and the interpretation of results. Include code cells that perform the analysis and produce the output. Use markdown cells to explain what each code cell does and why.

Step 3: Declare Dependencies

Create a dependency file that lists all software packages and versions required to run the notebook. For Python, use a requirements.txt file or an environment.yml file for conda. For R, record the session information including the R version and all package versions.

Step 4: Test the Notebook from a Clean Environment

Run the notebook from a clean environment to verify that it works without any hidden dependencies. This means creating a fresh environment, installing the declared dependencies, and running the notebook from top to bottom. Document any issues that arise and fix them.

Step 5: Version Control the Notebook and Dependencies

Initialize a Git repository and commit the notebook, dependency files, and documentation. Commit changes regularly with descriptive messages. Tag the version that corresponds to a publication or report.

Step 6: Share the Notebook and Supporting Files

Share the notebook along with the data, dependency files, and documentation through a public repository such as GitHub or a data repository. Provide clear instructions for how to set up the environment and run the notebook.

Records and Measurements for Reproducibility

Track Execution Order and Results

Notebooks can be executed in different orders, and the execution order can impact the results. A study of Jupyter notebooks found that testing different execution orders impacts reproducibility rates. Record the execution order and verify that the notebook produces the same results when run from top to bottom.

Document the Computing Environment

Record the operating system, processor architecture, and software versions used for the analysis. This information is essential for reproducing the analysis on a different machine. Include this information in the notebook or in a separate environment file.

Measure Reproducibility Success

Track the success rate of notebook reproduction attempts. The large-scale study of biomedical notebooks found that only a small fraction of notebooks ran through without errors. Use this information to identify common failure patterns and improve your own notebooks.

Record Data Versions

Data files change over time, and the version of the data used for an analysis affects the results. Record the version or hash of each data file used in the analysis. Store data files in a version-controlled repository or use a data versioning tool.

Common Failure Patterns in Notebook Reproducibility

Hidden State and Execution Order Dependencies

Notebooks can accumulate hidden state when cells are executed out of order. A cell may depend on variables defined in a cell that was executed earlier but is not visible in the current session. This makes the notebook difficult to reproduce because the execution order matters.

To avoid this problem, run the notebook from top to bottom in a fresh kernel before sharing it. Avoid defining variables in one cell and using them in a cell that appears earlier in the notebook.

Missing or Undeclared Dependencies

Notebooks often fail to reproduce because dependencies are not declared or are declared incorrectly. The large-scale study found that dependencies could not be installed for a substantial fraction of notebooks. Declare all dependencies explicitly and test the notebook from a clean environment.

Absolute File Paths

Notebooks that use absolute file paths fail when run on a different machine. Use relative paths or environment variables to refer to data files and output locations. Store data files within the project directory structure.

Hard-Coded Values and Magic Numbers

Notebooks that contain hard-coded values for parameters or thresholds are difficult to reproduce because the source of these values is unclear. Define parameters in a separate configuration file or at the top of the notebook with clear explanations.

Outdated Code and Deprecated Functions

Code that uses deprecated functions or outdated package versions may fail when run in a current environment. Use current versions of packages and update the notebook when dependencies change. Test the notebook regularly to ensure it still runs.

Large Output Files in the Notebook

Notebooks that contain large output files such as figures or tables become difficult to manage and share. Save output files to disk and reference them from the notebook instead of embedding them in the notebook file.

Quality Controls for Notebook Development

Code Review Practices

Review notebooks as part of the research process. A second person can identify issues that the original author missed, including unclear narrative, missing dependencies, or execution order problems. Code review is a standard practice in software engineering and applies equally to notebook development.

Automated Checks

Use automated tools to check notebook quality. The Julynter tool was designed based on findings and best practices from a study of 1.4 million notebooks from GitHub. Such tools can identify common issues including missing dependencies, execution order problems, and poor coding practices.

Continuous Integration Testing

Set up continuous integration testing to automatically run the notebook when changes are committed. This ensures that the notebook continues to work as the code and data evolve. Continuous integration is a standard practice in software engineering that applies to notebook development.

Documentation Standards

Follow documentation standards for notebooks. Use markdown cells to explain the research question, the data, the methods, and the interpretation. Include comments in code cells to explain complex operations. Document the computing environment and dependency versions.

Sharing Notebooks for Reproducible Research

Public Repositories

Share notebooks through public repositories such as GitHub. This allows others to inspect the code, report issues, and suggest improvements. Public repositories also provide version control and issue tracking.

Data Repositories

Share data files through data repositories that provide persistent identifiers and versioning. The Research Data Framework from the National Institute of Standards and Technology provides guidance for managing research data. Data repositories ensure that data files remain accessible over the long term.

Notebook Repositories

Some repositories are specifically designed for sharing notebooks. These repositories may provide features such as notebook preview, execution, and citation. Choose a repository that matches the needs of your research community.

Supplementary Materials

Supplementary information can enhance transparency, credibility, and reproducibility of research when used effectively. Key recommendations include creating a single well-structured self-contained supplementary information master document, ensuring explicit cross-referencing between the main text and supplementary information, making supplementary information machine-readable, and avoiding the misuse of supplementary information as a substitute for proper data repositories.

Publication Guidelines

Follow journal-specific guidelines for sharing code and data. Many journals require authors to deposit code and data in public repositories. The EQUATOR Network provides reporting guidelines for health research that may include requirements for computational reproducibility.

Limitations and Considerations

Programming Knowledge Requirements

Current notebook systems require programming knowledge, which limits their wider adoption by the research community. The GenePattern Notebook environment was developed to address this limitation by providing access to genomic tools without the need to write code. Researchers with limited programming experience may need training or assistance to use notebooks effectively.

Scale and Complexity

Notebooks may not be suitable for very large or complex analyses. Workflow management tools may be more appropriate for analyses that involve many steps, large data sets, or high-performance computing. The ioProc workflow manager provides an example of a tool designed for complex workflows.

Long-Term Preservation

Notebooks depend on software environments that change over time. Even with containerization, long-term preservation requires ongoing maintenance. Research institutions and data repositories may provide support for long-term preservation of computational workflows.

Training and Skills Development

Using notebooks effectively requires training and skills development. Workshops and training programs can increase competence and foster collaboration. The AI for Clinical Care workshop demonstrated significant increases in self-reported knowledge gain across learning objectives for biomedical professionals.

Professional Escalation Criteria

When to Seek Technical Support

Seek technical support when you encounter issues that you cannot resolve through documentation or troubleshooting. This includes dependency conflicts that cannot be resolved, containerization problems, or version control issues that affect the integrity of your analysis.

When to Consult a Data Management Specialist

Consult a data management specialist when your project involves large or complex data sets, when you need to preserve data for the long term, or when you need to comply with data management requirements from funders or journals. The Research Data Framework from the National Institute of Standards and Technology provides guidance for research data management.

When to Involve a Statistician or Methodologist

Involve a statistician or methodologist when your analysis involves complex statistical methods, when you need to assess the stability of your results, or when you are uncertain about the appropriate analysis approach. The predictability, computability, and stability framework provides guidance for assessing the reliability of data science results.

When to Escalate to Institutional Review

Escalate to institutional review when your research involves human subjects, sensitive data, or regulatory requirements. Institutional review boards and data protection officers can provide guidance on compliance requirements.

Frequently Asked Questions

What is the difference between Jupyter and R Markdown?

Jupyter is a notebook system that supports multiple programming languages including Python, R, and Julia. R Markdown is integrated into the R ecosystem and supports dynamic report generation. Both systems allow you to combine narrative text, code, and output in a single document. The choice between them depends on your programming language preference and the needs of your research community.

How do I declare dependencies for a Jupyter notebook?

For Python notebooks, declare dependencies in a requirements.txt file or use an environment manager such as conda with an environment.yml file. List all packages and their versions. Test the notebook from a clean environment to verify that the dependency file is complete and accurate.

How do I declare dependencies for an R Markdown document?

Record the R version and package versions using the sessionInfo function or use the renv package to manage R dependencies. Include this information in the R Markdown document or in a separate file. Test the document from a clean environment to verify that it runs without errors.

Why does my notebook fail to reproduce on another machine?

Common causes include missing or undeclared dependencies, absolute file paths, hidden state from execution order, and outdated code. Run the notebook from top to bottom in a fresh kernel, declare all dependencies explicitly, use relative paths, and test the notebook from a clean environment.

What is containerization and why is it important for reproducibility?

Containerization with Docker preserves the full computing environment including the operating system, software versions, and dependencies. This ensures that the analysis reproduces at a later point in time even if the computing platform or software has changed. Containers are important for long-term reproducibility because software updates can break analyses even when dependency files are preserved.

How do I version control a notebook?

Initialize a Git repository and commit the notebook, dependency files, and documentation. Commit changes regularly with descriptive messages. Tag the version that corresponds to a publication or report. Use Git to track changes to the notebook and to make earlier versions accessible.

What should I include when sharing a notebook?

Share the notebook along with the data, dependency files, and documentation. Provide clear instructions for how to set up the environment and run the notebook. Include information about the computing environment and data versions. Share through public repositories that provide persistent identifiers and versioning.

How do I know if my notebook is reproducible?

Run the notebook from a clean environment with only the declared dependencies installed. Execute the notebook from top to bottom and verify that it produces the expected results. Document the execution order and the computing environment. Test the notebook on a different machine to verify that it works in a different environment.

Related Articles

• Jupyter Notebooks in Reproducible Research: Strengths, Limits, and Better Habits
• Git for Research Projects: Version Control for Code, Data, and Analysis Notes
• Research Data Management Plan: From File Naming to Long-Term Sharing
• Research In Computational Molecular Biology
• Research In Computational Molecular Biology

References and Further Reading

• Research Data Framework. National Institute of Standards and Technology.
• EQUATOR Network. EQUATOR Network.
• Experimental Design Assistant. NC3Rs.
• NCBI Literature Resources. National Center for Biotechnology Information.
• PubMed. National Library of Medicine.
• Computational reproducibility of Jupyter notebooks from biomedical publications.. GigaScience, 2024.
• A simple kit to use computational notebooks for more openness, reproducibility, and productivity in research.. PLoS computational biology, 2022.
• Towards reproducible computational drug discovery.. Journal of cheminformatics, 2020.
• Issues in Reproducible Simulation Research.. Bulletin of mathematical biology, 2019.
• Reactive, reproducible, collaborative: computational notebooks evolve.. Nature, 2021.
• Understanding and improving the quality and reproducibility of Jupyter notebooks.. Empirical software engineering, 2021.
• Veridical data science.. Proceedings of the National Academy of Sciences of the United States of America, 2020.
• The GenePattern Notebook Environment.. Cell systems, 2017.
• Building AI competence in the healthcare workforce with the AI for clinical care workshop: A Bridge2AI for clinical CHoRUS project.. 2025.
• A framework for reproducibly managing coupled research software and data assets based on shared transformation functions.. 2026.
• Implementing a training resource for large-scale genomic data analysis in the All of Us Researcher Workbench.. 2025.
• Ten simple rules for making the supplement increase your paper's impact.. 2026.
• Launching an independent research laboratory in a digital era: practical lessons for early career investigators.. 2026.
• DynareR: Harnessing Dynare for Economic Modelling in R, R Markdown and Quarto. Computational Economics, 2024.
• Automating Clinical Trial Reporting with R Markdown and R Shiny. International Journal For Multidisciplinary Research, 2024.
• A Reproducible Data Analysis Workflow with R Markdown, Git, Make, and Docker. 2019.
• Upgrading an Old QCA Study to Make it More Transparent and Reproducible Using R Markdown
• Analyzing the reproducibility of research-related Jupyter notebooks at scale. Lecture Notes in Informatics Lni Proceedings Series of the Gesellschaft Fur Informatik Gi, 2025.
• ReproduceMeGit: A Visualization Tool for Analyzing Reproducibility of Jupyter Notebooks. Lecture Notes in Computer Science Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics, 2021.
• The computational notebook paradigm for multi-paradigm modeling. Proceedings 2019 ACM IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion Models C 2019, 2019.

This article is educational and does not replace institutional policy, professional advice, or applicable safety and regulatory requirements.