Common Mistakes When Sharing Research Code: Causes and Fixes

By Dr. Zubair Khalid, DVM, MS, PhD · 2026-11-11

Most shared research code fails before anyone even reaches it. In a review of 400 articles funded by the Nathan Shock Centers, 92 percent had no code availability statement at all, and only 6 percent pointed to a code repository [1]. Among articles that did promise open code, reviewers could still not locate the material in roughly one case out of five [1]. So the most likely cause of a code sharing failure is not a subtle technical error. It is a missing statement, a dead link, or a repository that exists but cannot run.

The first thing to check is simple. Open the paper, find the code availability statement, click the link, and try to run the entry point script on a clean machine. If any of those four steps fails, you have found the mistake, and the sections below explain which one it probably is and how to fix it.

This article walks through the common mistakes in sharing research code from the most frequent to the rarest, explains the mechanism behind each failure, gives a short diagnostic sequence you can follow, and ends with a prevention checklist for reproducible research.

Why Code Sharing Breaks Down in Practice

Three forces work against code sharing, and understanding them helps you fix the right problem rather than a symptom.

The first force is time. In a multidisciplinary survey of 254 researchers, the leading barriers to code sharing were lack of time to prepare documentation (65 percent), publication pressure (51 percent), and insufficient funding (42 percent) [2]. Researchers finish the analysis, submit the paper, and treat the code as an afterthought. The code that gets uploaded is the working directory from the last day of analysis, not a prepared artifact.

The second force is unclear expectations. Journal research data policies in materials science show that expectations for research software lag well behind expectations for data, with limited attention to versioning, persistent identifiers, dependency disclosure, and reproducible execution environments [3]. Among the largest cardiovascular research funders, data sharing mandates appear in 67 percent of policy documents, but requirements for code sharing are uncommon [4]. When no one clearly requires a runnable package, nobody prepares one.

The third force is timing. In radiological AI research, reviewers identified reporting as a final-stage manuscript requirement rather than a component embedded in study design, which causes methodological detail to be lost before anyone writes it down [5]. The same dynamic destroys code. A pipeline that existed in the analyst's head and terminal history in month one has evaporated by month twelve.

The payoff for fixing this is real. When economics and political science journals mandated data and code sharing, independent teams could computationally reproduce more than 85 percent of published claims across 110 articles [6]. Mandatory sharing policies convert code from a promise into a checkable artifact.

Troubleshooting Table: Symptom, Cause, Confirmation, Fix

The table below covers the failure modes a reader or reviewer hits most often. Work down it in order when you receive a complaint about your code, because the top rows are the most common.

The Mistakes, From Most Common to Rarest

Mistake 1: Never writing an availability statement

This is the dominant failure. In the Nathan Shock Centers review, 92 percent of articles had no code availability statement, and in real world evidence research published in one population data science journal, no article out of 41 included a link to the analytical code [1][10]. The mechanism is mundane. Availability statements are written at submission, and if the journal does not enforce one, the default is silence.

The fix is procedural, not technical. Decide at the start of the project where the code will live. Write the availability statement when you write the methods section, not the night before submission. A statement that names a specific repository and a specific archived version is the standard that assessment studies check against [1].

Mistake 2: "Available on request" as a substitute for sharing

Requests fail quietly. A reader emails the corresponding author, the email goes to an inbox that changed hands after a postdoc moved on, and nothing arrives. The Nathan Shock Centers study found that even among articles claiming open availability, materials could be located for only 81 percent of code claims [1]. The fix is to share the code, and to reserve "on request" for cases with genuine legal constraints, stating exactly what the constraint is.

Mistake 3: Dead links and unstable hosting

A GitHub link in a PDF can rot when an account is renamed or a repository goes private. Survey work on code sharing in clinical prediction model research found that even when repository links exist, the retrieved repositories vary enormously in reproducibility features [7]. Journal policies rarely require persistent identifiers for software, which is exactly the gap [3]. The fix is a two-layer approach. Keep the live repository for development, and deposit a snapshot in an archival service that issues a persistent identifier. Cite the archived version in the paper.

Mistake 4: Missing dependencies and unpinned versions

This is the most common technical failure once the code is found. A script that imports packages without a version file works on the author's machine and fails everywhere else, because package maintainers change interfaces over time. The TRIPOD-Code development team identified dependency specification and executable structure as core reproducibility features that most shared repositories lack [7].

The fix takes minutes if done during the project. In Python, freeze your environment into a requirements.txt or an environment.yml file. In R, use a lockfile that records exact package versions. State the language version as well. A survey of Global Burden of Disease secondary analyses checked whether articles even reported the R version they used, and treated that as a marker of analytical transparency [11], which tells you how often even this minimum step is skipped.

Mistake 5: No README, no documentation

The survey barrier data make the mechanism clear. Documentation is the single most time-consuming part of code sharing, cited by 65 percent of researchers as a barrier [2], so it is the first thing cut. The result is a repository full of scripts and no entry point.

The fix is a short README that answers five questions. What does this code produce. What must be installed. What input files are needed and where do they go. Which command reproduces the main result. Who do I contact when it breaks. In an experimental survey of computational reproduction, 85 independent teams tried to replicate one study's results, and teams given the original code succeeded far more often than teams given only a methods description, at 95.7 percent versus 89.3 percent for sign and significance, and 76.9 percent versus 48.1 percent for exact numerical matches to two decimal places [12]. Documentation is what converts a code dump into a usable reproduction package.

Mistake 6: Hardcoded paths and missing input data

Code that contains C:\Users\alice\Documents\project\ or /home/alib/ fails on every other machine. The mechanism is convenience during development. The fix is to make the project root the reference point for all paths, keep input files in a documented folder inside the project, and never reference locations outside it. When the underlying data cannot be shared, include a small synthetic dataset with the same column names and structure so the pipeline can be executed end to end. Studies using electronic health record text face exactly this constraint, and frameworks that generate deidentified analytic text preserving contextual information show one route to sharing something useful under privacy constraints [13].

Mistake 7: Nondeterministic output

If your analysis involves random number generation, model initialization, or resampling, an unseeded run produces different numbers each time, and a reader will conclude the code is broken. The fix is to set a seed at the top of every script that uses randomness, record software versions, and state in the README that outputs should match the published tables exactly. If outputs genuinely cannot be matched because a proprietary service changed, say so explicitly rather than leaving the reader to guess.

Mistake 8: No map from code to published results

A repository can run cleanly and still fail the reader, because nothing tells them which script produces Table 1, which produces Figure 3, and in what order the scripts must run. The scoping review behind TRIPOD-Code found substantial heterogeneity in reproducibility features across shared repositories and called for clearer expectations beyond mere code availability, including documentation and executable structure [7]. The fix is an output index in the README, script names that match their outputs, and a single master script or Makefile that runs the whole pipeline in order.

Mistake 9: Wrong version, no version

If the repository keeps evolving after publication, the code on the main branch next year will not be the code that produced the paper. Journal policy analysis shows versioning and persistent identifiers for software receive limited attention [3]. The fix is to tag the exact commit used for submission, archive that snapshot, and cite the archived identifier in the paper. Treat the live repository as a moving workspace and the archived snapshot as the frozen record.

Mistake 10: Closed models reported as if they were reproducible

Studies using large language models have a specific version of this problem. In a scoping review of 63 rheumatology studies using large language models, 79 percent relied on closed or proprietary systems, and reporting of basic transparency items such as the language of interaction was missing in about two thirds of studies [8]. A closed model can change under your feet between submission and publication. The fix is to report the model name, version, access dates, prompt text, and inference settings, and to prefer open-weight models when the comparison allows it. This mirrors a broader reporting gap in radiological AI, where limited usability of shared resources restricts independent verification even when resources are nominally shared [5].

Mistake 11: Assuming privacy rules block all sharing

Some teams respond to restricted data by sharing nothing, which conflates the data with the code. Analytical code usually contains no protected information and can almost always be shared. For the data itself, a tiered approach is now being proposed in clinical trial transparency, in which what you release should match the scientific claims it needs to support, ranging from reduced publication-time datasets to fuller reproducibility packages [9]. The fix is to share the code always, document the data access pathway, and release the highest data tier that governance permits.

Mistake 12: No license

A repository without a license is legally "all rights reserved," which means a reader technically cannot reuse it. The fix takes one click. Choose a permissive open license unless your institution requires otherwise, and state it in the README.

A Short Diagnostic Sequence

When you prepare code for sharing, or when someone reports a problem with code you shared, run this sequence in order.

1. Open the paper and locate the availability statement. If there is none, write and deposit one.
2. Click the link in a private browser window. Confirm the repository is public and complete.
3. Clone the repository to a clean machine or a fresh container. If installation fails, the dependency file is missing or wrong.
4. Run the entry point script. If it crashes on a path or a missing file, fix paths and include or generate the inputs.
5. Run it a second time. If the numbers differ between runs, find the missing seed or the unpinned version.
6. Compare the output to the published figure or table. If they diverge, identify the undocumented step.
7. Hand the repository to a colleague in your field who was not involved in the project. Time how long it takes them to reproduce one result. That number is your real usability metric.

This sequence mirrors what professional reproduction teams do. Large-scale reproduction studies found that even with mandatory sharing policies, robustness varied, and that more experienced reproducer teams uncovered more fragility [6]. Your colleague test is the affordable version of that scrutiny.

Prevention: Reproducible Research Practices and a Reproducibility Checklist

Expert consensus work through a Delphi consultation of 73 panel members from 34 countries prioritized reproducibility measures and interventions for implementation, reflecting broad agreement that verification practices should be built into research rather than patched on afterward [14]. The FAIR principles, which stand for Findable, Accessible, Interoperable, and Reusable, were developed for data and have since been extended to research software and workflows, and they provide the organizing logic for a code sharing checklist [15][16]. The recurring lesson from FAIR practice is that findable and accessible are easy, while interoperable and reusable require deliberate design [15].

Use this reproducibility checklist before every submission.

1. A code availability statement names a specific repository.
2. The repository is public, and the link was tested in a private browser window.
3. The exact version used for the paper is tagged and archived with a persistent identifier.
4. A README states what the code does, what to install, what inputs are needed, and which command to run.
5. A dependency file pins package versions, and the language version is stated.
6. All scripts use paths relative to the project root.
7. A synthetic or example dataset lets the pipeline run end to end when real data are restricted.
8. Every random process has a recorded seed.
9. A master script runs the full pipeline in the correct order.
10. An output index maps each script to each published figure and table.
11. Any proprietary model or API is reported with name, version, access dates, and settings [8].
12. A license file is present.
13. No credentials, API keys, or protected data are anywhere in the repository.
14. A colleague has reproduced at least one headline result using only the repository.

Educational resources built around executable, script-based workflows, such as open teaching materials for metabolomics that carry FAIR practice from raw data through to annotated results [16], and modular preprocessing frameworks in psychophysiology that generate automatic quality control reports [17], show what the finished artifact looks like. Tools like these demonstrate that a reproducibility package can double as a teaching resource, which increases the return on the time invested.

For high-throughput biology specifically, sharing analysis code alongside data is one of the core guidelines for avoiding irreproducible omics research, along with prespecified hypotheses, power analysis, and attention to batch effects [18]. In that setting the code is part of the evidence, because the analyst's choices about normalization, filtering, and multiple testing correction determine the result as much as the instrument does.

A Note on Security and Protected Data

Code repositories are a common source of accidental credential leaks. Before you make anything public, search the repository for API keys, passwords, database connection strings, and tokens, and move them into environment variables or a configuration file that is excluded from version control. If your code touches human data, confirm that no patient identifiers, dates of birth, or free-text clinical narratives remain in the repository, including in example outputs and notebooks. Removing these items after a public release does not fully remove them from the version history, so clean before the first public commit, not after.

Common Mistakes and Limitations

Even careful researchers repeat a handful of errors, and honest practice means acknowledging the limits of what sharing achieves.

Treating the repository as a dump rather than a deliverable. Uploading the working directory in the final hour produces a package with no README, no dependency file, and no output map. Budget time for code preparation the way you budget time for figure preparation.

Confusing availability with reproducibility. A runnable repository is necessary but not sufficient. The TRIPOD-Code effort exists precisely because code availability statements alone do not guarantee that code is documented, dependency-complete, and executable [7]. The reproducibility checklist above targets the gap between the two.

Assuming a clean run proves the result. Reproduction research shows that teams reproducing the same study produced varied workflows and encountered many small procedural variations and mistakes, and that transparency helped but did not eliminate the need for institutional checks and better training [12]. A successful run confirms computational reproducibility, not scientific correctness.

Overrelying on journal and funder pressure as the only mechanism. Journal requirements and institutional incentives were the highest-ranked supporting measures in the researcher survey [2], but policy analysis shows enforcement remains uncommon and software expectations lag behind data expectations [3]. Individual practice has to carry the load until policies catch up.

Forgetting that closed models are a moving target. If your results depend on a proprietary model, a perfect repository still cannot guarantee reproduction, which is why reporting the exact model version and settings is not optional [8].

Finally, a limitation of the evidence itself. Most of what we know comes from surveys and assessments in specific fields, including aging research, clinical prediction models, materials science journals, radiology AI, and social science [1][7][3][5][6]. The exact prevalence numbers differ by field and journal, so treat the percentages here as field-specific baselines rather than universal constants.

Frequently Asked Questions

What is the most common mistake when sharing research code?

The most common mistake is having no functional sharing pathway at all. In one assessment of aging research articles, 92 percent had no code availability statement, and even among articles promising open code, about one in five could not be located [1]. Fix that first, before polishing anything else.

What should a README for research code contain?

A README should state the purpose of the code, installation requirements, required input files and their locations, the exact command that reproduces the main results, expected runtime, and a contact for questions. Documentation is the most time-intensive part of code sharing according to researchers themselves, so keep it short but complete [2].

How do I share code when my data are confidential?

Share the code, which almost never contains protected information, and pair it with a small synthetic dataset that has the same structure so the pipeline runs end to end. Document the data access pathway and release the highest tier of data that your governance approval permits [9].

What is the difference between code availability and reproducibility?

Code availability means the code can be found and downloaded. Reproducibility means an independent person can run it and obtain your published numbers, which additionally requires documentation, pinned dependencies, seeds, and executable structure. Assessment studies treat these as distinct and find reproducibility features lag far behind availability [7].

Do journals require code sharing?

Some do, but requirements remain uneven. In materials science, nearly all journals have a research data policy, yet public deposition is rarely mandatory and expectations for software are notably less developed than for data [3]. Funder mandates for code sharing are also uncommon compared with data sharing mandates [4].

How do I make my Python or R project reproducible?

Pin your package versions in a lockfile, state the language version, set random seeds, use paths relative to the project root, add a master script that runs everything in order, and test the whole thing on a clean machine or fresh container. These steps address the dependency and execution gaps most often found in shared repositories [7].

Does sharing code actually improve science?

Yes, and there is quantitative evidence. In 110 economics and political science articles published under mandatory data and code sharing policies, independent teams could computationally reproduce more than 85 percent of published claims, and 72 percent of statistically significant estimates held up under robustness checks [6]. Sharing turns claims into checkable artifacts.

How much time should I budget for preparing code for release?

There is no fixed number, but plan for it as a real task spanning days, not an hour. The dominant reported barriers are lack of time to prepare documentation, publication pressure, and insufficient funding [2]. Preparing the repository as you work, rather than at the end, keeps the final effort small and the checklist above becomes a quick confirmation rather than a reconstruction project.

Sources

1. An assessment of data and code sharing in research funded by the Nathan Shock Centers, 2017-2022: a meta-research study.
2. Exploratory Insights into Barriers and Open Practices for Computational Reproducibility in Scientific Research.
3. Journal research data policies in materials science.
4. Funders' Expectations for Open Science in Cardiovascular Research: A Scoping Review of the Largest Cardiovascular Research Funders.
5. Transparent reporting is central to reproducible radiological AI research: A call to action.
6. Reproducibility and robustness of economics and political science research.
7. AI-assisted scoping review of code sharing in clinical prediction model research.
8. Large language models in rheumatology-centered research: A scoping review of model openness and reporting practices.
9. Clinical trial data transparency should match data access to scientific purpose: a tiered framework for reproducibility and reuse.
10. Extent of Open Science Practices in the Reporting of Real World Evidence Research.
11. Paper mill or paper mine? A tentative answer to the sharp increase in research papers based on the Global Burden of Disease database.
12. The reliability of replications: a study in computational reproductions.
13. Balancing Privacy Protection and Reproducible Entity Extraction in Real-World Data in the Development of a Framework for Deidentified Japanese Electronic Health Record Narratives Using BERT: Retrospective Observational Study.
14. Prioritising measures and interventions to strengthen research reproducibility: a Delphi consultation study.
15. Introducing the Truth Effect Database (TED): An open trial-level resource promoting FAIR data in truth effect research.
16. Metabonaut: an open educational resource for learning reproducible, script-based untargeted LC-MS/MS metabolomics.
17. eyeris: A Flexible, Extensible, and Reproducible Pupillometry Preprocessing Framework in R.
18. Promises and Pitfalls of High-Throughput Biological Assays.