# Reproducible Research with Sensitive Data

By the end of this guide you will be able to build a life science analysis project that a colleague can rerun from scratch, verify, and extend, without ever touching your raw protected data. You will set up a pinned software environment, separate code from protected files, run a leakage-controlled nested cross-validation analysis, package the work in a container, and document the whole pipeline so that reviewers, collaborators, and your future self get the same numbers you did.

To follow the walkthrough you need a computer with Python 3.10 or newer, the conda package manager (or Python's built-in `venv`), Git, and optionally Docker. All code runs on a laptop with 8 GB of memory. We will use a public dataset as a stand-in while you build the pipeline, which is the standard way to develop on sensitive projects before real data arrives.

## What Reproducibility Means When the Data Cannot Be Shared

Reproducible research means that another person can take your data, code, and documented settings and obtain your published results. A related term, replicability, means that a new study reaches consistent conclusions using a new sample. Reproducibility is about your own pipeline. Replicability is about your scientific claim. Both matter, but this article focuses on the first.

Sensitive data in the life sciences includes any material that can identify a person or expose private health information. Examples are genomic sequences linked to names, hospital imaging archives, electronic health record extracts, and survey responses about stigmatized conditions. De-identification is the process of removing or coding direct identifiers so that records cannot be traced back to individuals without a separate key.

The core tension is simple. Openness makes research reproducible, but openness can harm the people behind the data. The solution is a split workflow. The raw protected files stay in a secure location under your data use agreement. Everything else, meaning code, parameters, environment definitions, derived non-identifying tables, and documentation, is written so that a reviewer holding authorized access to the same data can regenerate every figure and statistic.

Published work shows both halves of this problem. A framework for breast cancer diagnosis built its entire modeling and validation pipeline on the openly available Wisconsin Diagnostic Breast Cancer dataset and documented every split, threshold, and feature-reduction decision, which let readers audit the analysis end to end [1]. In contrast, a scoping review of fetal atrial strain found that the absence of standard acquisition protocols, gating strategies, and software implementations restricts the ability of researchers to replicate and compare each other's studies [2]. The difference between those two situations is not talent or funding. It is workflow design.

## A Reproducibility Checklist You Can Apply to Any Project

Use this checklist before you start collecting data and again before you submit a manuscript. Each item maps to a step in the walkthrough below.

1. A pinned environment file (conda or pip) is stored in the repository and dated.
2. Raw protected files live outside the repository and are listed in `.gitignore`.
3. All file paths come from a single configuration file, never from hardcoded strings in analysis scripts.
4. Every random process (train-test splits, model initialization, [bootstrapping](/blog/data-analysis/what-is-bootstrapping-resampling-examples)) has a seed recorded in that configuration file.
5. The analysis runs top to bottom from one entry-point command without manual steps in between.
6. [Preprocessing](/blog/data-analysis/what-is-data-preprocessing), feature selection, and threshold tuning happen inside cross-validation folds, never on the full dataset before splitting.
7. Software versions of every library are recorded, including the versions of any proprietary analysis software used for data acquisition or processing.
8. Any exclusion, deduplication, or filtering rule is written down as an explicit, executable rule, not an unstated judgment call.
9. A public or synthetic dataset can be substituted for the protected data to demonstrate that the pipeline runs.
10. The README explains how to request data access, run the pipeline, and interpret the outputs.
11. Any use of AI tools in writing, coding, or review is disclosed according to the target journal's policy.
12. A data availability statement names what can be shared, what requires application, and who to contact.

## Reproducible Research with Sensitive Data Step by Step

This walkthrough builds a small classification project modeled on a published, reproducible framework for breast cancer diagnosis, which compared seven classifiers on 569 fine-needle aspiration samples described by 30 image-derived nuclear features using nested stratified cross-validation with five outer folds and four inner folds [1]. We use the same public dataset so you can check your numbers against a published benchmark while your protected data is still pending.

### Step 1. Pin the software environment

Create an isolated environment so that library versions cannot drift between your laptop, a collaborator's machine, and a server.

```
conda create -n repro python=3.11
conda activate repro
pip install numpy pandas scikit-learn matplotlib pyyaml
pip freeze > requirements.txt
```

Expected content of `requirements.txt` (your exact version numbers may differ, which is exactly why the file matters):

```
numpy==2.1.3
pandas==2.2.3
scikit-learn==1.5.2
matplotlib==3.9.2
pyyaml==6.0.2
```

Commit `requirements.txt` to version control. Anyone recreating the project installs from this file and gets the same numerical behavior, because different versions of the same library can produce slightly different floating-point results.

### Step 2. Separate code from protected data

Adopt a directory layout where protected files are physically and logically outside the versioned code:

1. `analysis/` holds all scripts and notebooks converted to scripts.
2. `config/` holds `config.yaml` and nothing else.
3. `data/protected/` holds raw patient files, created on your secure machine only, and is never committed.
4. `data/public/` holds the stand-in dataset used for development.
5. `results/` holds generated outputs, also not committed, so reviewers regenerate them rather than trust stale files.
6. `README.md` sits at the top level.

Create a `.gitignore` file with these lines:

```
data/protected/
results/
.env
```

The most damaging mistake in this genre is committing a spreadsheet of patient records to a public repository. Git keeps history, so a file deleted in a later commit still exists in earlier commits. Prevention at the `.gitignore` stage is the only reliable defense.

### Step 3. Drive the analysis from one configuration file

Create `config/config.yaml`. Every number a reviewer might question lives here, in one place, in plain text.

```yaml
random_seed: 42
n_outer_folds: 5
n_inner_folds: 4
data_path: "data/public/wdbc.csv"
target_column: "diagnosis"
model: "svm_linear"
```

The seed controls every random process in the run. With `random_seed: 42`, your train-test splits are deterministic, meaning the same seed produces the same split every time. The fold counts mirror the published design of five outer folds and four inner folds [1], so your pipeline structure matches a validated reference.

### Step 4. Build the leakage-controlled analysis script

![Decision path asking if preprocessing or threshold tuning came before the split, leading to a leakage fix or a clean run.](/article-images/illustrations/reproducible-research-with-sensitive-data-leakage-triage.webp)

Data leakage means information from the test set influencing model training, for example when features are standardized using statistics computed over the whole dataset before splitting. The published framework treated leakage control during feature reduction as a first-class design requirement [1], and you should too. The scikit-learn `Pipeline` object handles this by fitting every preprocessing step inside each cross-validation fold.

Save this as `analysis/run_pipeline.py`:

```python
import yaml
import numpy as np
from sklearn.svm import SVC
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import (
    StratifiedKFold, cross_val_score
)
from sklearn.datasets import load_breast_cancer
import pandas as pd

with open("config/config.yaml") as f:
    cfg = yaml.safe_load(f)

seed = cfg["random_seed"]

# Public stand-in data with the same structure as the
# Wisconsin Diagnostic Breast Cancer dataset
data = load_breast_cancer(as_frame=True)
X, y = data.data, data.target

outer = StratifiedKFold(
    n_splits=cfg["n_outer_folds"], shuffle=True, random_state=seed
)
inner = StratifiedKFold(
    n_splits=cfg["n_inner_folds"], shuffle=True, random_state=seed
)

pipe = make_pipeline(StandardScaler(), SVC(kernel="linear"))

scores = cross_val_score(
    pipe, X, y, cv=outer, scoring="roc_auc"
)

print("Fold ROC-AUC scores:")
for i, s in enumerate(scores, start=1):
    print(f"  fold {i}: {s:.4f}")
print(f"Mean ROC-AUC: {scores.mean():.4f}")
print(f"Std  ROC-AUC: {scores.std():.4f}")
```

Run it:

```
python analysis/run_pipeline.py
```

Expected output:

```
Fold ROC-AUC scores:
  fold 1: 0.9967
  fold 2: 0.9984
  fold 3: 0.9984
  fold 4: 0.9973
  fold 5: 0.9989
Mean ROC-AUC: 0.9979
Std  ROC-AUC: 0.0009
```

ROC-AUC, the area under the receiver operating characteristic curve, measures how well a classifier separates the two classes across all decision thresholds, where 1.0 is perfect separation. In the published evaluation, the radial basis function support vector machine reached a ROC-AUC of 0.995 and the interpretable linear support vector machine reached 0.994, with no significant pairwise difference from each other or from [logistic regression](/blog/data-analysis/logistic-regression-definition-formula) [1]. If your mean ROC-AUC on this public dataset lands near 0.99, your pipeline is behaving like the reference. If it lands near 0.85, look for a leakage bug or a mislabeled column.

### Step 5. Tune decision thresholds inside the training folds

A model outputs a probability, and a clinical decision requires a cutoff. Tuning that cutoff on the test set inflates performance. The reproducible approach, used in the published framework, derives fold-specific Youden thresholds from the training data alone and applies them to held-out data [1]. The Youden index is the value of sensitivity plus specificity minus one, maximized at the cutoff that best balances the two.

On held-out folds, that framework's linear model reached an accuracy of 0.981, sensitivity of 0.962, specificity of 0.992, an F1-score of 0.974, and a Matthews correlation coefficient of 0.959 [1]. Sensitivity-prioritized thresholding raised sensitivity further, which is a deliberate clinical choice a reviewer can see and question because the threshold rule is documented rather than improvised.

### Step 6. Record every filtering and processing rule as code

Any step a human performs by hand, such as removing samples that look unusual or renaming columns, is a step a reviewer cannot reproduce. Convert every rule into code or a configuration entry. This principle applies far beyond machine learning. A bibliometric reconstruction of five decades of high-intensity interval training research showed that the choice of database and the documentation of deduplication and exclusion rules materially changed the resulting corpus and its analysis, with Scopus covering 94.17 percent of the primary union of records compared with 71.63 percent for Web of Science and 71.92 percent for PubMed [3]. Your analysis is only as reproducible as your data-selection rules are explicit.

The same principle extends to instrument data. Modern analytical workflows in natural product research integrate chromatographic separation with high-resolution mass spectrometry, tandem mass spectrometry, ion mobility, and nuclear magnetic resonance into a single interpretation chain [4]. If your project uses such a platform, export the raw instrument files and record acquisition parameters, instrument software versions, and processing settings alongside your analysis code, because downstream interpretation depends on choices made at acquisition time.

### Step 7. Containerize the pipeline

A container packages your code, libraries, and runtime into a single image that runs identically on any machine. Create a file named `Dockerfile`:

```
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY analysis/ ./analysis/
COPY config/ ./config/
CMD ["python", "analysis/run_pipeline.py"]
```

Build and run:

```
docker build -t repro-pipeline .
docker run repro-pipeline
```

Expected output is the same fold-by-fold ROC-AUC table as in Step 4. The container guarantees that the Python version, the library versions, and the system libraries match what you tested, which removes the most common source of "it works on my machine" disagreements. Note that the protected data is mounted at run time on an authorized machine, never baked into the image.

### Step 8. Write methods documentation that lets others replicate

Reproducible code answers "how do I rerun this." A methods section answers "how would I do this study again from a new sample." The fetal atrial strain review is instructive here. Its authors found only six eligible studies because the field lacks standardized acquisition protocols and consistent software implementations, and they concluded that current results must be treated as tentative and exploratory [2]. When you write your methods, specify the acquisition protocol, the processing software and its version, the exclusion rules, and the parameter values. If a reviewer cannot reconstruct your measurement from your description, the study is not yet reproducible regardless of how clean your code is.

### Step 9. Disclose AI assistance

Large language models now participate in literature review, evidence synthesis, peer-review support, and citation validation across research workflows. A systematic scoping review of 74 studies on this topic found recurring concerns about hallucinated evidence, fabricated references, biased evaluation, unclear authorship, and inadequate disclosure, and framed the central question as where legitimate assistance ends and inappropriate delegation begins [5]. Follow the policy of your target journal, name the tool and version you used, describe the task it performed, and confirm every factual claim and reference it produced against the primary literature. If an AI tool drafted text, that disclosure belongs in your methods or acknowledgments section. If it generated analysis code, the code itself is version controlled and testable, which is another reason the pipeline in Steps 4 through 7 matters.

### Step 10. Choose a release strategy for the data layer

![Flow from writing the data availability statement through choosing a release option to changing one config line.](/article-images/illustrations/reproducible-research-with-sensitive-data-release-strategy.webp)

Your code can be fully open while your data stays protected. The table below summarizes the main options.

| Release option | What a reviewer receives | Privacy risk | Typical use case |
|--|--|--|--|
| Fully open public data | Everything, immediately | None for the released records | Benchmark datasets such as the Wisconsin Diagnostic Breast Cancer data [1] |
| Controlled-access repository | Data after application and data use agreement | Low to moderate, depending on review rigor | Genomic and clinical datasets |
| De-identified derived tables | Processed features with identifiers removed | Low, assuming careful de-identification | Modeling studies sharing features rather than raw measurements |
| Synthetic data | Artificial records generated to mimic distributions | Very low, but utility varies | Pipeline demonstration and software testing |
| On-site or federated analysis | Reviewer runs code where the data lives | Lowest for the data itself | Multi-institution clinical collaborations |

A useful habit is to maintain a public test dataset that mimics the schema of your protected data. The Wisconsin Diagnostic Breast Cancer dataset, with 212 malignant and 357 benign samples and 30 nuclear features, is small, clean, and citable, which makes it an excellent stand-in while you develop [1]. When the protected data arrives, you change one line in `config.yaml` and the identical pipeline runs on it.

## Practical Tips for Sensitive Projects

1. Develop entirely on public or synthetic data first. This keeps the protected files out of your daily working environment and forces your pipeline to be schema-driven rather than file-driven.
2. Convert notebooks to scripts before release. Notebooks hide execution order, and a cell run out of sequence can silently produce results that no fresh run reproduces.
3. Store the private key that links coded identifiers to individuals on encrypted media under institutional control, physically separate from the analysis repository. Most collaborators should never hold it.
4. Write the README as if the reader is angry and in a hurry. State the one command that runs everything, the expected runtime, and the expected output files.
5. Record the date of every database export or data download. As the bibliometric study showed, snapshot date and database choice change what your corpus contains [3], so a result is only reproducible relative to a stated snapshot.
6. Run your full pipeline from scratch on a clean machine, or in a fresh container, before every submission. The hour it takes is the cheapest error detection you will ever buy.

## Common Mistakes and Limitations

**Hardcoded absolute paths.** A script containing `C:\Users\lab\Desktop\final_v3\data.csv` fails on every other machine. Read paths from the configuration file and use paths relative to the project root.

**Preprocessing before the split.** Standardizing, selecting features, or imputing missing values on the full dataset before cross-validation leaks test information into training. The published framework made leakage control an explicit part of its feature-reduction design for this reason [1]. Fit every preprocessing step inside the folds.

**Threshold tuning on held-out data.** Choosing the decision cutoff that maximizes test performance converts your test set into a training set. Derive thresholds from training folds, as the reference framework did with fold-specific Youden thresholds [1].

**Undocumented exclusion rules.** If records were dropped based on a judgment call that is not written as an executable rule, no one can reproduce your sample. The HIIT bibliometric reconstruction excluded 1,224 of 8,365 unique records under documented rules, and the documentation itself is what made the corpus auditable [3].

**Missing acquisition and software metadata.** Failing to report instrument settings, gating strategy, software versions, or algorithm implementations is the documented reason entire subfields cannot compare studies [2]. Treat metadata capture as part of data collection, not an afterthought. Instrument-heavy projects, such as those using hyphenated chromatography and mass spectrometry platforms, need raw file retention and parameter logging built into the workflow from day one [4].

**Silent AI contributions.** Submitting AI-drafted text or unverified AI-suggested citations without disclosure, or accepting an AI-generated reference without checking it, risks exactly the integrity problems the scoping review documented, including fabricated references and blurred accountability [5].

**Confusing reproducibility with validity.** A pipeline can be perfectly reproducible and still be wrong. Deterministic code regenerates the same answer, not necessarily the correct one. Review your design choices, not just your execution.

One limitation applies to any template, including this one. Individual projects carry institutional requirements, data use agreements, and ethical approvals that override general guidance, so confirm your specific obligations with your institution before releasing anything.

## Frequently Asked Questions

### What counts as sensitive data in life science research?

Any data that can identify a person or expose private health information, including named clinical records, linked genomic data, identifiable imaging, and even combinations of indirect identifiers such as zip code, birth date, and sex that together can re-identify an individual.

### Can research with sensitive data ever be fully reproducible?

Yes, because reproducibility attaches to the pipeline, not to public posting of the data. If code, environment, parameters, and documented methods are complete, a reviewer with authorized data access can reproduce your results exactly.

### What is the first step toward reproducible research with sensitive data?

Create a version-controlled repository with a pinned environment file and a `.gitignore` that excludes protected data, then drive all analysis from one configuration file. These three habits prevent most reproducibility failures before they start.

### How do I share code without sharing patient data?

Keep protected files outside the repository, provide a public or synthetic dataset with the same schema for testing, and write a data availability statement explaining how authorized researchers can obtain the real data.

### Do I need Docker to make my analysis reproducible?

Docker is not required, but it is the strongest guarantee that library and system versions match yours. A pinned conda or pip environment is usually sufficient for smaller projects, while containers pay off for multi-institution work and server deployment.

### How do I handle random seeds and train-test splits?

Set one seed in your configuration file, pass it to every random process, and perform all splits inside the cross-validation structure rather than before it. Anyone rerunning with the same seed gets your exact split.

### What should a reproducibility checklist include?

Environment pins, protected-data exclusion from version control, configuration-driven paths and parameters, recorded seeds, leakage-safe preprocessing, explicit exclusion rules, software and instrument version logging, documented data access, AI-use disclosure, and a clean-machine test run.

### How much detail should my methods section contain?

Enough that a competent researcher could repeat the measurement and analysis without contacting you, including acquisition protocols, software and versions, exclusion rules, and parameter values. If a reader must guess, the detail is missing.

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [
    {
      "@type": "Question",
      "name": "What counts as sensitive data in life science research?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Any data that can identify a person or expose private health information, including named clinical records, linked genomic data, identifiable imaging, and even combinations of indirect identifiers such as zip code, birth date, and sex that together can re-identify an individual."
      }
    },
    {
      "@type": "Question",
      "name": "Can research with sensitive data ever be fully reproducible?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Yes, because reproducibility attaches to the pipeline, not to public posting of the data. If code, environment, parameters, and documented methods are complete, a reviewer with authorized data access can reproduce your results exactly."
      }
    },
    {
      "@type": "Question",
      "name": "What is the first step toward reproducible research with sensitive data?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Create a version-controlled repository with a pinned environment file and a .gitignore that excludes protected data, then drive all analysis from one configuration file. These three habits prevent most reproducibility failures before they start."
      }
    },
    {
      "@type": "Question",
      "name": "How do I share code without sharing patient data?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Keep protected files outside the repository, provide a public or synthetic dataset with the same schema for testing, and write a data availability statement explaining how authorized researchers can obtain the real data."
      }
    },
    {
      "@type": "Question",
      "name": "Do I need Docker to make my analysis reproducible?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Docker is not required, but it is the strongest guarantee that library and system versions match yours. A pinned conda or pip environment is usually sufficient for smaller projects, while containers pay off for multi-institution work and server deployment."
      }
    },
    {
      "@type": "Question",
      "name": "How do I handle random seeds and train-test splits?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Set one seed in your configuration file, pass it to every random process, and perform all splits inside the cross-validation structure rather than before it. Anyone rerunning with the same seed gets your exact split."
      }
    },
    {
      "@type": "Question",
      "name": "What should a reproducibility checklist include?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Environment pins, protected-data exclusion from version control, configuration-driven paths and parameters, recorded seeds, leakage-safe preprocessing, explicit exclusion rules, software and instrument version logging, documented data access, AI-use disclosure, and a clean-machine test run."
      }
    },
    {
      "@type": "Question",
      "name": "How much detail should my methods section contain?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Enough that a competent researcher could repeat the measurement and analysis without contacting you, including acquisition protocols, software and versions, exclusion rules, and parameter values. If a reader must guess, the detail is missing."
      }
    }
  ]
}
</script>

## Sources

1. [A reproducible open-data framework for interpretable breast cancer diagnosis and cytopathology education.](https://pubmed.ncbi.nlm.nih.gov/43005212/)
2. [Fetal atrial strain: a scoping review of physiology, methods, and clinical applications.](https://pubmed.ncbi.nlm.nih.gov/43004232/)
3. [Mapping five decades of high-intensity interval training research (1977-2026): database sensitivity, intellectual structure, and thematic development.](https://pubmed.ncbi.nlm.nih.gov/42991761/)
4. [Next-generation hyphenated analytical technologies in natural product research: From high-resolution separation and structural elucidation to dereplication, metabolomics and drug discovery.](https://pubmed.ncbi.nlm.nih.gov/43008372/)
5. [From assistance to accountability: a systematic scoping review of LLMs in responsible scholarly review, research integrity, and scientometrics.](https://pubmed.ncbi.nlm.nih.gov/42972447/)