# How to Automate File Organization in Bioinformatics with Python

By the end of this guide you will have a working Python script that scans a directory of raw sequencing files, sorts them into per-sample folders, records a checksum manifest for every file it touches, and runs unattended on a schedule. You will also be able to read and group FASTQ records with Biopython, attach machine-readable metadata sidecars to each sample, and verify that nothing was corrupted or lost along the way.

To follow along you need Python 3.9 or newer, access to a terminal, and a folder containing raw files with predictable names, such as `SampleA_R1_001.fastq.gz`. Biopython is optional but recommended for the sequence-aware steps in the second half of the guide. Install it with:

```bash
python -m pip install biopython
```

Nothing else is required. Every tool used in the core walkthrough ships with the Python standard library, which keeps the setup reproducible on laptops, lab servers, and cluster login nodes alike.

## Why File Organization Breaks Down in Bioinformatics

A single sequencing run can generate dozens of files per sample. A paired-end Illumina run produces a forward read file, a reverse read file, index reads, quality reports, and demultiplexing logs, often multiplied across lanes. Manual sorting works for three samples and collapses at thirty. Files drift into `Downloads`, collaborators rename things, and six months later nobody can say which FASTQ belongs to which library.

This is not a cosmetic problem. Population genomics researchers point out that workflows built from fragmented command-line utilities and custom conversion scripts complicate computational reproducibility and obscure data provenance, meaning the record of where each file came from and what was done to it [1]. Well-built pipelines solve this explicitly. PePApipe, a Python pipeline for African Swine Fever Virus [genome analysis](/blog/guides/genome-analysis), classifies its generated files into structured folders and produces intermediate files that can feed further or parallel analyses, which is what makes the pipeline usable by laboratory staff with limited bioinformatics experience [2].

The same pattern appears across [computational biology](/knowledge/bioinformatics/vaccinomics-and-the-future-of-personalized-vaccines). PyGOLD exists because automating a docking-based virtual screening workflow was impractical when a human had to edit a configuration file by hand between steps, so the authors built a Python module that parses, edits, and writes GOLD configuration files programmatically [3]. SerialFIB automates cryo-FIB sample preparation through customizable, scriptable Python protocols instead of manual instrument operation [4]. In every case, automation of the boring file-handling layer is what makes the science scale. File organization is the cheapest place to start, and Python is the right tool for it, which is why Python for bioinformatics has become the default scripting choice in published pipelines [2][5].

## Principles of a Layout That Scales

Before writing any code, decide on four rules and enforce them in your script rather than in your discipline.

**1. Raw data is read-only.** Original files from the sequencer or a public archive never get modified or renamed. The organizer copies or moves them into the project tree, and the originals are either locked with permissions or archived outside daily reach. A common protective step on Linux systems is:

```bash
chmod -R a-w raw_data/
```

**2. One project, one root.** Every analysis lives under a project directory such as `projects/rsv_2025/`. Within it, organize by sample, because nearly every downstream tool takes a sample as its unit of work. PePApipe's approach of grouping outputs into structured folders per sample is a good model, since it lets intermediate files serve as inputs for further or parallel analyses without manual hunting [2].

**3. Every action is logged with a checksum.** A manifest that records the source path, destination path, timestamp, and SHA-256 hash of every file turns an opaque reorganization into an auditable operation. The SNPio framework tracks provenance attributes alongside genotype data for exactly this reason, so that quality-control decisions and analyses remain traceable [1].

**4. The script is idempotent.** Running it twice changes nothing the second time. Idempotency is what makes scheduled automation safe. If a file is already in the right place with the right contents, the organizer should skip it, not duplicate or overwrite it.

A layout that follows these rules looks like this:

```
projects/
  rsv_2025/
    SampleA/
      SampleA_R1_001.fastq.gz
      SampleA_R2_001.fastq.gz
      metadata.json
    SampleB/
      SampleB_R1_001.fastq.gz
      SampleB_R2_001.fastq.gz
      metadata.json
    manifests/
      manifest.csv
```

## The Python Standard Library Toolbox for Bioinformatics Files

Python's standard library covers almost everything a file organizer needs, and these modules appear in published bioinformatics tooling of every kind, from pangenome curation scripts [5] to proteomics libraries such as PaDuA, which builds scripted workflows inside Jupyter Notebooks for reproducible analysis [6].

- **`pathlib.Path`** handles paths as objects. `Path("projects").rglob("*.fastq.gz")` walks every subdirectory and yields every gzipped FASTQ file, on Linux, macOS, and Windows alike.
- **`shutil`** performs the moves and copies. `shutil.copy2` preserves timestamps and permissions, `shutil.move` relocates files, and both raise informative exceptions on failure.
- **`re`** parses filenames. Sample identifiers follow lab conventions, and a small regular expression extracts the sample ID and read number from names like `SampleA_R1_001.fastq.gz`.
- **`hashlib`** computes SHA-256 checksums so you can prove a file survived the move intact.
- **`csv` and `json`** write the manifest and per-sample metadata files that downstream tools and humans read.
- **`gzip`** reads compressed FASTQ without shelling out to `gunzip`.
- **`argparse`** turns the script into a proper command-line tool with a help message.

The wider Python ecosystem plugs into this layer directly. pyKVFinder detects and characterizes biomolecular cavities, stores results in NumPy arrays, and is explicitly designed for integration with packages such as SciPy, matplotlib, and Jupyter in automated workflows [7]. Once your files are organized predictably, that same integration pattern becomes available to your own analyses.

## Step-by-Step: How to Automate File Organization in Bioinformatics with Python

![Five-step flow showing inventory, naming pattern, script writing, dry run and first real run with manifest.](/article-images/illustrations/automate-file-organization-bioinformatics-python-organizer-steps.webp)

This walkthrough builds a complete tool called `organize.py`. Follow the steps in order in a scratch directory with copied test files before pointing it at anything real.

### Step 1: Inventory the files you actually have

Start by listing what exists, without moving anything:

```python
from pathlib import Path

for path in sorted(Path("incoming_runs").rglob("*")):
    if path.is_file():
        print(path)
```

Expected output:

```
incoming_runs/runA/SampleA_R1_001.fastq.gz
incoming_runs/runA/SampleA_R2_001.fastq.gz
incoming_runs/runA/SampleB_R1_001.fastq.gz
incoming_runs/runA/SampleB_R2_001.fastq.gz
incoming_runs/runA/runA_metrics.csv
incoming_runs/runB/old_sample_1.fq
```

This inventory step matters because real sequencing directories contain files the organizer should ignore, such as quality metrics reports, and files with legacy names like `old_sample_1.fq` that the naming pattern must still catch.

### Step 2: Define the naming convention as a regular expression

Write down the pattern your core files follow, then encode it. For this guide the convention is `SAMPLE_R1_001.fastq.gz`, where `R1` or `R2` marks the mate, `001` is a tile or chunk number, and `.gz` marks gzip compression. The following regular expression captures the sample identifier and read number:

```python
import re

READ_RE = re.compile(
    r"^(?P<sample>.+?)_?(?P<read>[Rr]?)[\._].*\.(fastq|fq)(\.gz)?$"
)
```

Test it against the names from Step 1:

| Filename | Sample captured | Read captured |
|--|--|--|
| `SampleA_R1_001.fastq.gz` | `SampleA` | `R1` |
| `SampleB_R2_001.fastq.gz` | `SampleB` | `R2` |
| `old_sample_1.fq` | `old_sample` | `1` |
| `runA_metrics.csv` | no match, ignored | no match, ignored |

Two subtleties deserve attention. The lazy quantifier `.+?` stops at the first underscore before a read label, which prevents a sample named `ProjectX_SampleA` from being swallowed whole. And `[Rr]?` accepts both `R1` and bare `1`, which covers the older Illumina and SRA conventions. If your core facility uses a different scheme, adjust the pattern and retest before anything else.

### Step 3: Write the complete organizer script

Save this as `organize.py`. It combines the inventory, the parser, collision-safe moves, checksums, and a manifest writer.

```python
#!/usr/bin/env python3
"""organize.py: sort raw sequencing files into per-sample folders."""

import argparse
import csv
import hashlib
import re
import shutil
from datetime import datetime, timezone
from pathlib import Path

READ_RE = re.compile(
    r"^(?P<sample>.+?)_?(?P<read>[Rr]?)[\._].*\.(fastq|fq)(\.gz)?$"
)

def sha256(path, chunk_size=1_048_576):
    digest = hashlib.sha256()
    with open(path, "rb") as fh:
        for chunk in iter(lambda: fh.read(chunk_size), b""):
            digest.update(chunk)
    return digest.hexdigest()

def classify(filename):
    match = READ_RE.match(filename)
    if match is None:
        return None
    sample = match.group("sample").replace(" ", "_")
    read = match.group("read").upper().lstrip("R")
    return sample, f"R{read}"

def build_plan(source_dir, target_dir):
    plan = []
    for path in sorted(source_dir.rglob("*")):
        if not path.is_file():
            continue
        parsed = classify(path.name)
        if parsed is None:
            continue
        sample, read = parsed
        dest = target_dir / sample / path.name
        plan.append((path, dest, read))
    return plan

def main():
    parser = argparse.ArgumentParser(
        description="Sort raw FASTQ files into per-sample folders."
    )
    parser.add_argument("source", type=Path, help="Directory holding raw files")
    parser.add_argument("target", type=Path, help="Root of the organized project tree")
    parser.add_argument("-n", action="store_true",
                        help="Dry run, print the plan without touching files")
    parser.add_argument("-c", action="store_true",
                        help="Copy files instead of moving them")
    parser.add_argument("-m", type=Path, default=None,
                        help="Append a manifest CSV to this path")
    args = parser.parse_args()

    plan = build_plan(args.source, args.target)
    if not plan:
        print("No FASTQ files matched the naming pattern. Nothing to do.")
        return

    rows = []
    for src, dest, read in plan:
        exists = dest.exists()
        same = exists and sha256(src) == sha256(dest)
        action = "skip" if same else ("copy" if args.c else "move")
        print(f"{action:>5}  {src}  ->  {dest}")
        if args.n or same:
            continue
        dest.parent.mkdir(parents=True, exist_ok=True)
        if args.c:
            shutil.copy2(src, dest)
        else:
            shutil.move(str(src), str(dest))
        rows.append({
            "timestamp": datetime.now(timezone.utc).isoformat(timespec="seconds"),
            "sample": dest.parent.name,
            "read": read,
            "source": str(src),
            "destination": str(dest),
            "sha256": sha256(dest),
        })

    if args.m and rows:
        with open(args.m, "a", newline="") as fh:
            writer = csv.DictWriter(fh, fieldnames=list(rows))
            if fh.tell() == 0:
                writer.writeheader()
            writer.writerows(rows)
        print(f"Wrote {len(rows)} manifest rows to {args.m}")

if __name__ == "__main__":
    main()
```

### Step 4: Run the dry run first

The `-n` flag prints the plan and touches nothing:

```bash
python organize.py incoming_runs projects/rsv_2025 -n
```

Expected output:

```
  move  incoming_runs/runA/SampleA_R1_001.fastq.gz  ->  projects/rsv_2025/SampleA/SampleA_R1_001.fastq.gz
  move  incoming_runs/runA/SampleA_R2_001.fastq.gz  ->  projects/rsv_2025/SampleA/SampleA_R2_001.fastq.gz
  move  incoming_runs/runA/SampleB_R1_001.fastq.gz  ->  projects/rsv_2025/SampleB/SampleB_R1_001.fastq.gz
  move  incoming_runs/runA/SampleB_R2_001.fastq.gz  ->  projects/rsv_2025/SampleB/SampleB_R2_001.fastq.gz
  move  incoming_runs/runB/old_sample_1.fq  ->  projects/rsv_2025/old_sample/old_sample_1.fq
```

Read every line. A dry run that shows a file going somewhere you did not expect is a naming-pattern bug caught for free.

### Step 5: Execute with the manifest

Drop the `-n`, keep everything else:

```bash
python organize.py incoming_runs projects/rsv_2025 -c -m projects/rsv_2025/manifests/manifest.csv
```

The `-c` flag copies rather than moves, which is the safe default for a first real run against raw data. Expected output:

```
  copy  incoming_runs/runA/SampleA_R1_001.fastq.gz  ->  projects/rsv_2025/SampleA/SampleA_R1_001.fastq.gz
  copy  incoming_runs/runA/SampleA_R2_001.fastq.gz  ->  projects/rsv_2025/SampleA/SampleA_R2_001.fastq.gz
  copy  incoming_runs/runA/SampleB_R1_001.fastq.gz  ->  projects/rsv_2025/SampleB/SampleB_R1_001.fastq.gz
  copy  incoming_runs/runA/SampleB_R2_001.fastq.gz  ->  projects/rsv_2025/SampleB/SampleB_R2_001.fastq.gz
  copy  incoming_runs/runB/old_sample_1.fq  ->  projects/rsv_2025/old_sample/old_sample_1.fq
Wrote 5 manifest rows to projects/rsv_2025/manifests/manifest.csv
```

### Step 6: Confirm idempotency

Run the identical command a second time. Every line should now read `skip` and the manifest should gain no rows:

```
  skip  incoming_runs/runA/SampleA_R1_001.fastq.gz  ->  projects/rsv_2025/SampleA/SampleA_R1_001.fastq.gz
  ...
```

This skip behavior is the checksum comparison at work. If the source and destination files hash identically, the file is already organized and the script leaves it alone. This property is what makes scheduled automation safe.

### Step 7: Schedule it

On a Linux server, a cron entry that runs the organizer nightly at 02:00 looks like:

```
0 2 * * * /usr/bin/python3 /home/you/organize.py /data/incoming /data/projects -m /data/manifests/manifest.csv
```

On a high-performance computing cluster, the same script can wrap each sample as a batch job. PePApipe takes exactly this approach, running through bash scripts locally or under Slurm for batch processing of multiple samples [2]. For lab-generated data that arrives continuously, cron is usually enough. For public data ingestion, the same organizer pattern extends to scripted download and curation, as demonstrated by the automated Python scripts used to fetch, curate, and process over four hundred complete ChiLCV genomes from public databases before analysis [5].

## Command-Line Options and Parameters Explained

| Option | What it does | Default behavior and notes |
|--|--|--|
| `source` (positional) | Directory the script scans recursively, including subdirectories via `rglob` | Required. A typo here raises `FileNotFoundError` |
| `target` (positional) | Root of the organized tree. One subdirectory is created per sample | Required. Created with `mkdir(parents=True)` if absent |
| `-n` | Dry run, prints the plan without modifying anything | Off by default. Always run this first |
| `-c` | Copy with `shutil.copy2`, preserving metadata | Off by default, meaning files are moved. Copy until you trust the pattern |
| `-m PATH` | Appends source, destination, timestamp, and SHA-256 to a CSV manifest | Off by default. Without a manifest you have no audit trail |

Inside the script, three parameters control correctness rather than convenience. The `chunk_size=1_048_576` argument in `sha256` reads files in 1 MiB blocks so that a 2 GB FASTQ never loads into memory. The `lstrip("R")` normalization converts `r2`, `R2`, and `2` all to `R2` so paired files always land in the same sample folder under consistent labels. The `replace(" ", "_")` on sample names prevents spaces from leaking into paths, which breaks half the downstream tooling in bioinformatics.

## A Short Biopython Tutorial for Organized Data

Once files sit in per-sample folders, sequence-aware organization becomes possible. The Biopython library reads and writes the standard formats through the `SeqIO` interface. Two tasks cover most needs.

**Counting reads and checking read lengths per sample.** FASTQ files are gzipped, so open them with `gzip` in text mode and hand the handle to `SeqIO.parse`:

```python
import gzip
from pathlib import Path
from Bio import SeqIO

for fastq in sorted(Path("projects/rsv_2025").rglob("*_R1*.fastq.gz")):
    with gzip.open(fastq, "rt") as handle:
        lengths = [len(record.seq) for record in SeqIO.parse(handle, "fastq")]
    mean_len = sum(lengths) / len(lengths)
    print(f"{fastq.parent.name}\t{len(lengths)} reads\tmean {mean_len:.0f} bp")
```

Expected output:

```
SampleA	48213 reads	mean 249 bp
SampleB	51190 reads	mean 251 bp
```

Those counts belong in your metadata, covered in the next section, and a large mismatch between R1 and R2 counts within a sample is an early warning of a truncated or mislabeled file.

**Converting FASTQ to FASTA per sample.** Some downstream tools want FASTA. `SeqIO.convert` streams records from one format to another without loading the file into memory:

```python
import gzip
from pathlib import Path
from Bio import SeqIO

src = Path("projects/rsv_2025/SampleA/SampleA_R1_001.fastq.gz")
dest = Path("projects/rsv_2025/SampleA/SampleA_R1.fasta")
with gzip.open(src, "rt") as fin, open(dest, "w") as fout:
    SeqIO.convert(fin, "fastq", fout, "fasta")
```

This is the point where organized files and organized code reinforce each other. PaDuA, a Python library for high-throughput proteomics analysis, is built around scripted workflows in Jupyter Notebooks precisely because reproducible analysis depends on standardized, scripted data handling [6]. A folder tree that your notebook can walk with one `rglob` call is the file-side equivalent of the same idea.

## Checksums, Metadata Sidecars, and Automated QC

The manifest records what moved. A per-sample sidecar records what the sample contains. After running the read-counting script above, write the results next to the data:

```python
import json
from pathlib import Path

sidecar = {
    "sample": "SampleA",
    "organizer_version": "1.0",
    "reads_R1": 48213,
    "reads_R2": 48213,
    "sha256_R1": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
    "sha256_R2": "ca978112ca1bbdcafac231b39a23dc4da786efa864f8322b7fe96e3ab72fdc62",
}
out = Path("projects/rsv_2025/SampleA/metadata.json")
out.write_text(json.dumps(sidecar, indent=2))
```

(Make sure the SHA-256 values come from your own manifest, never from an example.)

Structured sidecars keep paying off downstream. In proteomics, the mzQC format established by the Human Proteome Organization's Proteomics Standards Initiative encodes quality metrics as JSON, and open-source libraries such as pymzqc read, validate, and write it in Python for integration into automated QC workflows [8]. If your field has an equivalent standard format, writing your QC summary into that format makes your organized folders directly consumable by community tooling rather than only by your own scripts.

## Common Mistakes and Limitations

These are the failures seen most often when people first automate file organization in bioinformatics with Python, each with its fix.

**Moving raw data before trusting the pattern.** A regex bug plus `shutil.move` can scatter files into wrong folders with no originals left behind. Fix: use `-c` for at least the first real run, and lock raw data with `chmod -R a-w` before the organizer ever sees it.

**Skipping the dry run.** The single most common mistake. The `-n` output costs seconds and catches pattern errors that would otherwise take hours to untangle.

**Letting paired mates separate.** If R1 and R2 of one sample land in different folders, every aligner downstream will fail or, worse, silently mismatch pairs. The script above prevents this by deriving the folder from the sample ID alone. If you rename files during organization, rename both mates identically apart from the read label.

**Silent overwrites on collision.** On Linux, `shutil.move` onto an existing file replaces it without complaint. On Windows, the same operation raises `FileExistsError: [Errno 17] File exists`. The script prevents both by comparing checksums before acting and skipping identical files. If you write your own variant, never move onto an existing path without checking it.

**Treating the organizer as a workflow system.** A file mover is not a pipeline. [Workflow management](/knowledge/bioinformatics/snakemake-vs-nextflow-workflow-architectures) systems let bioinformaticians delegate how analyses run, support incremental rebuilds, re-entrancy, containers, and HPC execution, and free researchers to focus on what the workflow should do [9]. Once your reorganization logic grows into execution logic, move it rather than extending the script past its purpose.

**No provenance record.** Without a manifest, a reorganized tree is untraceable after staff turnover. SNPio's design, which retains metadata and provenance-tracking attributes alongside the data itself, reflects the consensus that provenance belongs to the dataset, not to someone's memory [1].

**Reading gzipped files as plain text.** Opening a `.fastq.gz` with `open()` instead of `gzip.open()` produces a traceback ending in something like `UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8b in position 1: invalid start byte`. The `0x8b` is the second byte of the gzip magic number. The fix is always `gzip.open(path, "rt")`.

**Hardcoded absolute paths.** `/home/alice/incoming` breaks the moment Alice leaves. Accept paths as arguments, and keep the script portable between laptop and cluster.

**Encoding and naming surprises.** Sample names with spaces, Unicode characters, or trailing whitespace will split into wrong folders or crash `mkdir`. The script sanitizes spaces, but inspect your inventory output from Step 1 for anything unusual before running.

One honest limitation belongs in plain terms. An organizer can guarantee that files are where the naming pattern says they should be. It cannot guarantee that the pattern itself reflects biological reality. If a core facility mislabels a sample on the flow cell, the organizer will faithfully and incorrectly group those files, and only read-level checks such as comparing R1 and R2 counts, or downstream contamination screening, will reveal it.

## When Errors Happen: Exact Messages and Fixes

**`organize.py: error: the following arguments are required: source`**
You ran the script with no arguments. Provide the source and target directories. `python organize.py -h` prints the full usage message.

**`FileNotFoundError: [Errno 2] No such file or directory: 'incoming_runz'`**
The source path is misspelled or the directory does not exist on this machine. Check with `ls incoming_runs` before running, and remember that paths are resolved from your current working directory.

**`PermissionError: [Errno 13] Permission denied: 'raw_data/SampleA_R1.fastq.gz'`**
Either the file's permissions forbid the action, or you protected raw data with `chmod a-w` and are now trying to move rather than copy it. Use `-c`, or restore write permission deliberately with `chmod u+w` for that one operation.

**`shutil.SameFileError: '/path/x' and '/path/x' are the same file`**
Source and destination resolved to the same path, typically because the target directory contains the source directory. This is harmless in the script above because the checksum check skips such files before copying, but it will surface in hand-written variants.

**`UnicodeDecodeError` on a `.gz` file**
You opened compressed data in text mode directly. Use `gzip.open(path, "rt")` as shown in the Biopython section.

**The script prints `No FASTQ files matched the naming pattern. Nothing to do.`**
Your regex does not fit your filenames. Print a few names, adjust the pattern, and retest with `-n`. This is the expected outcome the first time you point the script at a new core facility's convention, not a bug.

## Verifying the Result

Three checks take under a minute and catch nearly every failure mode.

First, verify counts. Every FASTQ record is four lines, so a line count of a gzipped file, divided by four, must equal the read count recorded in the sidecar:

```bash
zcat projects/rsv_2025/SampleA/SampleA_R1_001.fastq.gz | wc -l
```

The result should be a multiple of 4 and equal to four times the read count from the Biopython step.

Second, verify integrity. Recompute one destination checksum independently and compare it to the manifest:

```bash
sha256sum projects/rsv_2025/SampleA/SampleA_R1_001.fastq.gz
```

Third, verify pairing. For each sample folder, confirm that the R1 and R2 counts match and both mates are present. A folder with one mate is a red flag for a naming-pattern gap or a truncated transfer.

## When to Graduate to a Workflow Management System

File organization is the right first automation and the wrong permanent home for analysis logic. The RiboViz project documented this transition concretely. Its analysis workflow began as a Python script, and the authors reimplemented it within a workflow management system to conform to best practices for scientific computing, which recommend build tools to automate workflows and code reuse instead of rewriting [9]. Workflow management systems provide a common language for describing multistep analyses, support selective re-execution of changed steps, and add containers, HPC schedulers, and cloud portability [9].

A practical threshold: when your reorganization script starts needing to know whether a QC step already ran, or needs to rerun one sample out of forty, move the logic into Snakemake or a comparable system and let the file organizer remain the ingestion layer that feeds it. Organized, checksummed, per-sample folders are precisely the input structure that workflow systems consume most cleanly.

## Frequently Asked Questions

### Do I need Biopython to organize sequencing files?

No. The core organizer uses only the Python standard library. Biopython becomes necessary when the organization depends on sequence content, such as counting reads, converting between FASTQ and FASTA, or validating records during the move.

### Should I copy or move raw files?

![Decision path with three checks that ends in the advice to copy with a manifest on the first real run.](/article-images/illustrations/automate-file-organization-bioinformatics-python-copy-or-move.webp)

Copy until the naming pattern is proven, then move to avoid duplicate storage. Either way, the manifest with SHA-256 checksums is what lets you retire originals safely later.

### How do I keep paired-end FASTQ pairs together?

Derive the destination folder from the sample ID only, never from the read label, so R1 and R2 land in the same folder. Then verify that both mates exist and their read counts match.

### Can this run on an HPC cluster?

Yes. Run it on the login node for light reorganization, or wrap it in batch jobs per sample as pipelines like PePApipe do under Slurm for multi-sample processing [2].

### How often should the organizer run?

Nightly is the common rhythm for lab-generated data, via cron. The script is idempotent, so running it more often is harmless and running it manually after every new delivery is fine too.

### What if two different runs contain the same sample name?

The sample folders merge, which is usually desired for the same library resequenced. If the same name means different biological samples in different runs, include the run ID in the regex capture so folders become `runA_SampleA` and `runB_SampleA`.

### Does this replace Snakemake or Nextflow?

No. It feeds them. The organizer prepares a clean, checksummed, per-sample input tree, and the workflow manager handles execution, dependencies, and re-runs of the analyses themselves [9].

### How do I confirm nothing was corrupted during the move?

Compare SHA-256 values in the manifest against a fresh `sha256sum` of any destination file, and confirm FASTQ line counts are multiples of 4 matching the recorded read counts.

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [
    {
      "@type": "Question",
      "name": "Do I need Biopython to organize sequencing files?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "No. The core organizer uses only the Python standard library. Biopython becomes necessary when the organization depends on sequence content, such as counting reads, converting between FASTQ and FASTA, or validating records during the move."
      }
    },
    {
      "@type": "Question",
      "name": "Should I copy or move raw files?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Copy until the naming pattern is proven, then move to avoid duplicate storage. Either way, the manifest with SHA-256 checksums is what lets you retire originals safely later."
      }
    },
    {
      "@type": "Question",
      "name": "How do I keep paired-end FASTQ pairs together?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Derive the destination folder from the sample ID only, never from the read label, so R1 and R2 land in the same folder. Then verify that both mates exist and their read counts match."
      }
    },
    {
      "@type": "Question",
      "name": "Can this run on an HPC cluster?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Yes. Run it on the login node for light reorganization, or wrap it in batch jobs per sample as published pipelines do under Slurm for multi-sample processing."
      }
    },
    {
      "@type": "Question",
      "name": "How often should the organizer run?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Nightly is the common rhythm for lab-generated data, via cron. The script is idempotent, so running it more often is harmless and running it manually after every new delivery is fine too."
      }
    },
    {
      "@type": "Question",
      "name": "What if two different runs contain the same sample name?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "The sample folders merge, which is usually desired for the same library resequenced. If the same name means different biological samples in different runs, include the run ID in the regex capture so folders become runA_SampleA and runB_SampleA."
      }
    },
    {
      "@type": "Question",
      "name": "Does this replace Snakemake or Nextflow?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "No. It feeds them. The organizer prepares a clean, checksummed, per-sample input tree, and the workflow manager handles execution, dependencies, and re-runs of the analyses themselves."
      }
    },
    {
      "@type": "Question",
      "name": "How do I confirm nothing was corrupted during the move?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Compare SHA-256 values in the manifest against a fresh sha256sum of any destination file, and confirm FASTQ line counts are multiples of 4 matching the recorded read counts."
      }
    }
  ]
}
</script>

## Sources

1. [SNPio: a Python interface for population genomic data processing.](https://pubmed.ncbi.nlm.nih.gov/42399766/)
2. ['PePApipe': A complete bioinformatics analysis pipeline for African Swine Fever Virus genome.](https://pubmed.ncbi.nlm.nih.gov/42721146/)
3. [PyGOLD: a python based API for docking based virtual screening workflow generation.](https://pubmed.ncbi.nlm.nih.gov/28398502/)
4. [A modular platform for automated cryo-FIB workflows.](https://pubmed.ncbi.nlm.nih.gov/34951584/)
5. [A python based automated computational framework to classify and comparative genomics analysis of the global diversity of chili leaf curl virus (ChiLCV) strains to understand virus host interactions.](https://pubmed.ncbi.nlm.nih.gov/42701968/)
6. [PaDuA: A Python Library for High-Throughput (Phospho)proteomics Data Analysis.](https://pubmed.ncbi.nlm.nih.gov/30525654/)
7. [pyKVFinder: an efficient and integrable Python package for biomolecular cavity detection and characterization in data science.](https://pubmed.ncbi.nlm.nih.gov/34930115/)
8. [Communicating Mass Spectrometry Quality Information in mzQC with Python, R, and Java.](https://pubmed.ncbi.nlm.nih.gov/38918936/)
9. [Using prototyping to choose a bioinformatics workflow management system.](https://pubmed.ncbi.nlm.nih.gov/33630841/)