# How to Convert Bash Script to Nextflow

By the end of this guide you will have taken a real, working bash pipeline (quality control, adapter trimming, and alignment) and converted it into a Nextflow workflow that runs on your laptop, submits jobs to an HPC cluster, resumes after a crash, and ships each step in its own container. You will know how to read the run log, how to debug the failures that almost always appear on a first conversion, and how to decide whether Nextflow or Snakemake is the better fit for your project.

Before you start, have the following on hand:

- A Unix-like shell (Linux, macOS, or Windows Subsystem for Linux)
- Java 17 or later, since the Nextflow launcher is a JVM application
- A container engine such as Docker, Singularity, or Apptainer, installed and tested
- The bash script you want to convert, plus a small test dataset (two or three samples is enough)
- Roughly 20 GB of free disk space for Nextflow's `work` directory on a first run

A note on the bigger picture before the commands. [Computational biology](/knowledge/bioinformatics/vaccinomics-and-the-future-of-personalized-vaccines) has been moving for years from hand-run scripts toward formal, declarative descriptions of analysis pipelines, and introductory treatments of that shift emphasize a shared formalism that lets experimentalists and computational scientists talk about the same analysis [1]. Structured, tiered roadmaps for computational methods in biomedicine make the same argument from a different angle: foundations, core techniques, and applications each need explicit definition before they can be audited [2]. Converting a bash script to Nextflow is the practical, file-level version of that argument. Your script already encodes a workflow. Nextflow just makes the workflow explicit, inspectable, and restartable.

## Why Convert a Bash Script to a Bioinformatics Workflow Manager

A bash script that processes five samples by looping over them has several failure modes that get worse as the sample count grows.

1. **No restart.** If step 47 fails on sample 12 of 200, you comment out lines, hand-edit loops, and rerun everything.
2. **No parallelism model.** You write your own `&` and `wait` logic, or you run one sample at a time and lose days.
3. **Hidden dependencies.** The script assumes a specific PATH, specific conda environment, and specific genome index version. None of that is written down.
4. **No audit trail.** Six months later, nobody can say which version of the code, containers, and reference files produced a given BAM file.

A bioinformatics workflow manager fixes all four by making each step a named task with declared inputs and outputs, by hashing those tasks so unchanged work can be skipped, and by delegating scheduling to the platform (SLURM, SGE, Kubernetes, or AWS Batch). The Nextflow documentation and the nf-core community describe these as the core benefits of the framework, and the same ideas show up across every serious pipeline effort in the field.

There is also an institutional argument. Studies of digital transformation in small and medium organizations consistently find that the hard part is not the technology but people and process, and that structured training modules and clear implementation plans are what make adoption stick [3]. Health services research on large-scale practice change reaches the same conclusion from the clinical side, pairing an evidence-based program with a deliberate implementation guide and an enabling environment. Converting your pipeline is exactly this kind of planned practice change. Budget time for your labmates to learn the new format, and the conversion succeeds. Treat it as a one-weekend rewrite, and it usually stalls.

Finally, structured workflows support transparency obligations that are increasingly explicit in the literature. Frameworks for trustworthy public health data science now name fairness, transparency, and privacy protection as operational requirements across the data life cycle, from design through analysis and interpretation [4]. A Nextflow pipeline, with its pinned containers and logged execution traces, is the simplest concrete artifact you can show a reviewer or collaborator who wants to know exactly what ran.

## Nextflow vs Snakemake

![Two yes or no questions about many samples and Python use, leading to Nextflow or Snakemake as the recommended workflow manager.](/article-images/illustrations/convert-bash-script-to-nextflow-nextflow-or-snakemake.webp)

Both tools solve the same problem, and both are excellent. The honest differences:

| Aspect | Nextflow | Snakemake |
|--|--|--|
| Language | Groovy-based DSL | Python-based DSL |
| Execution model | Dataflow channels, tasks run as soon as inputs exist | DAG evaluated from target rules, jobs scheduled topologically |
| Container support | First-class, per-process Docker, Singularity, Conda | First-class, per-rule `use container` and `use conda` |
| Resume | Built-in, task-level caching, `nextflow run -resume` | Built-in, `snakemake, rerun-incomplete` with mtime-based triggers |
| Cluster backends | SLURM, SGE, LSF, PBS, Kubernetes, AWS Batch, Google Life Sciences | SLURM, SGE, LSF, PBS, Kubernetes, via generic cluster plugin |
| Ecosystem | nf-core, a large curated collection of peer-reviewed pipelines | Smaller but active ecosystem, strong in reproducibility papers |
| Best fit | Teams running many samples through fixed pipelines, cloud batch work | Groups already deep in Python, complex rule-based logic |

Choose Nextflow if your workload is "apply this series of tools to many samples and scale it out," which describes most production bioinformatics. Choose Snakemake if your analysis is logic-heavy, uses many Python steps, or if your group already thinks in Python. Both tools are far better than a 400-line bash script. Do not let the choice delay the conversion. The learning curves are comparable, and research on how students and professionals actually learn technical material shows that conceptual reorganization during training is non-linear, with an early phase where everything feels fragmented before the pieces integrate [5]. Expect that phase regardless of which manager you pick, and plan a small practice project before the real conversion.

## Core Nextflow Concepts You Need First

Nextflow runs version 2 of its language, called DSL2, which is the default since version 22.10. Four concepts cover most of what you need.

**Process.** A `process` is one step of the pipeline, the Nextflow equivalent of one logical block of your bash script. It declares inputs, outputs, a container, and a script body that is executed in a job sandbox.

**Channel.** A `channel` is an asynchronous sequence of data items that feeds processes. Channels are how Nextflow expresses parallelism. A channel holding 200 sample tuples becomes 200 parallel process executions without you writing a loop. Channel operations behave like functional programming: you map, filter, and combine streams without mutating them, a pattern that functional language tutorials describe as composing behavior from small, independent transformations [6].

**Workflow.** A `workflow` block is the calling layer. It builds channels and invokes processes in the order data flows, not the order lines appear.

**Work directory and publishDir.** Nextflow runs every task in a hash-named directory under `work/`. Nothing touches your project folder unless a process declares `publishDir`. This is the single biggest mental shift from bash, where the script writes directly into `results/`.

One more piece of vocabulary: the **executor**. The executor is the thing that actually runs each task, such as `local` on your laptop or `slurm` on a cluster. You change it in the config file, not in the pipeline code, which is why a well-converted Nextflow pipeline runs unchanged on a laptop and a 5,000-core cluster.

## The Running Example: A Bash QC and Alignment Pipeline

Here is a typical pre-Nextflow bash script. It is correct, it works, and it has every problem described above.

```bash
#!/bin/bash
# run_pipeline.sh
# Usage: bash run_pipeline.sh samples.txt
set -euo pipefail

SAMPLES_FILE=$1
mkdir -p results/fastqc results/trimmed results/aligned logs

while IFS=$'\t' read -r sample fq1 fq2; do
  echo "Processing $sample"
  fastqc "$fq1" "$fq2" -o results/fastqc

  cutadapt \
    -a AGATCGGAAGAGC -A AGATCGGAAGAGC \
    -o results/trimmed/${sample}_R1.trim.fq.gz \
    -p results/trimmed/${sample}_R2.trim.fq.gz \
    "$fq1" "$fq2" \
    > logs/${sample}.cutadapt.log

  hisat2 -x reference/grch38_index \
    -1 results/trimmed/${sample}_R1.trim.fq.gz \
    -2 results/trimmed/${sample}_R2.trim.fq.gz \
    -S results/aligned/${sample}.sam \
    2> logs/${sample}.hisat2.log

  samtools sort -@ 4 -o results/aligned/${sample}.bam \
    results/aligned/${sample}.sam
  samtools index results/aligned/${sample}.bam
  rm results/aligned/${sample}.sam
done < "$SAMPLES_FILE"

multiqc results/fastqc results/trimmed -o results/multiqc
echo "Done"
```

The script reads a three-column sample sheet (`sample`, forward FASTQ, reverse FASTQ), runs FastQC, trims adapters with Cutadapt, aligns with HISAT2, sorts and indexes with SAMtools, and produces a MultiQC report. Four tools, one loop, shared scratch directories, and zero restart capability. Converting this is the canonical "how to convert bash script to Nextflow" exercise, and the rest of the guide does exactly that.

## Convert Bash Script to Nextflow Step by Step

The walkthrough below assumes the bash script and sample sheet live in a directory called `myproject/` and that your test data sits in `data/`.

### Step 1. Inventory the Bash Script

Before writing any Nextflow, annotate the script. For each block of work, write down four things:

1. What it consumes (FASTQ files, the genome index)
2. What it produces (`*_fastqc.zip`, trimmed FASTQs, sorted BAMs, the MultiQC report)
3. What it writes into shared directories (here, `results/` and `logs/`)
4. Any state it mutates outside the loop (the `mkdir` calls, the index build)

This inventory is your process list. In this example it yields five processes: FASTQC, CUTADAPT, HISAT2_ALIGN, SAMTOOLS_SORT (which you can merge with the index call), and MULTIQC. Groups that skip this step routinely end up with one giant Nextflow process that is bash-with-extra-steps, which forfeits most of the benefit. The same lesson appears in industrial training contexts: the value comes from decomposing the work into teachable, inspectable modules, not from renaming the monolith.

### Step 2. Install Nextflow and Verify the Toolchain

```bash
curl -s https://get.nextflow.io | bash
sudo mv nextflow /usr/local/bin/
nextflow -v
```

Expected output:

```
nextflow version 24.10.0.5928
```

(Your patch version will differ. Any release from the last two years defaults to DSL2.)

Then verify the container engine, because you will attach one container per process:

```bash
docker run hello-world
# or, on HPC:
singularity --version
```

If you have never used containers, budget an hour for that first. Converting a script into a Nextflow pipeline whose tools are not actually pinned is half a conversion, and the pinning is where the reproducibility payoff lives. Published computational tutorials in adjacent fields now make "all code and reproducible experiments publicly available" a stated deliverable rather than an afterthought, and containerized Nextflow is the standard route to that deliverable in genomics.

### Step 3. Create the Skeleton `main.nf`

Create `main.nf` next to your bash script:

```groovy
#!/usr/bin/env nextflow
nextflow.enable.dsl = 2

process FASTQC {
    tag "${meta.id}"

    input:
    tuple val(meta), path(reads)

    output:
    tuple val(meta), path("*_fastqc.zip"), emit: zips

    script:
    """
    fastqc ${reads} -o .
    """
}

process CUTADAPT {
    tag "${meta.id}"

    input:
    tuple val(meta), path(reads)

    output:
    tuple val(meta), path("*_R1.trim.fq.gz"), emit: trimmed_r1
    tuple val(meta), path("*_R2.trim.fq.gz"), emit: trimmed_r2

    script:
    """
    cutadapt \\
        -a AGATCGGAAGAGC -A AGATCGGAAGAGC \\
        -o ${meta.id}_R1.trim.fq.gz \\
        -p ${meta.id}_R2.trim.fq.gz \\
        ${reads} ${reads[9]}
    """
}

workflow {
    // channels go here in Step 5
}
```

Two things in this skeleton trip up bash veterans, so read them closely.

First, the double backslash (`\\`) inside the `script` block. The script body is a Groovy string, and Groovy interpolates `${...}` before bash ever sees it. When you want bash to receive a literal `$variable`, escape the dollar sign. When you want Nextflow to substitute a Groovy value such as `${meta.id}`, leave it unescaped. Getting this backwards produces either an empty string or a bash error about an undefined variable, and it is the most common single bug in first conversions.

Second, `path(reads)` declares the input as files, which tells Nextflow to stage copies into the task's work directory. Never hard-code absolute paths inside a script body. The task directory is the working directory when the script runs, so relative filenames are the rule.

### Step 4. Convert One Process End to End Before Writing the Others

A convert-bash-script-to-nextflow tip that saves hours: do not convert all five steps at once. Convert FASTQC, run it, and confirm the output before touching CUTADAPT. Make the workflow block call only FASTQC for now, using a hardcoded test channel:

```groovy
workflow {
    ch_reads = channel.fromPath("data/{}_1.fq.gz", checkIfExists: true)
        .map { r1 -> tuple([[id: r1.getSimpleName()##_R1.readSimpleName() ], r1]) }
}
```

That version is fiddly, so the cleaner production pattern uses a sample sheet, which is exactly what your bash script already consumed. That is Step 5.

### Step 5. Build Channels from the Sample Sheet

Delete the temporary channel and replace the workflow block with:

```groovy
include { FASTQC } from './modules/fastqc.nf'
include { CUTADAPT } from './modules/cutadapt.nf'

workflow {
    ch_samples = Channel.fromPath(params.samplesheet, checkIfExists: true)
        .splitCsv(header: true, sep: '\t')
        .map { row -> tuple([id: row.sample], file(row.fq1, checkIfExists: true), file(row.fq2, checkIfExists: true)) }
        .map { meta, r1, r2 -> tuple(meta, [r1, r2]) }

    ch_fastqc = FASTQC(ch_samples).zips
    ch_trimmed = CUTADAPT(ch_samples)

    ch_fastqc.view()
}
```

And a `samples.csv` (or keep the tab-separated format and adjust `sep:`):

```
sample,fq1,fq2
S1,data/S1_R1.fq.gz,data/S2_R2.fq.gz
```

(The second row above is intentionally written with the S2 reads to show that the mapping, not the filename, determines the pairing. In your real sheet, keep the pairs consistent, and validate them. A mismatched R1 and R2 is a classic silent failure that surfaces only at alignment time as absurdly low mapping rates.)

Each row of the CSV becomes one tuple flowing down the channel. With two sample rows, FASTQC executes twice concurrently on a local executor, or as two queued jobs on SLURM. You have replaced the bash `while read` loop with a dataflow, and that replacement is the heart of the whole conversion. Many analysts coming from scripted analyses in other languages recognize the pattern immediately, since the same "load, transform, operate" mental model underlies introductory computational analysis curricula [7].

Run it:

```bash
nextflow run main.nf --samplesheet samples.csv -profile docker
```

Expected console output:

```
N E X T F L O W  ~  version 24.10.0
Launching `main.nf` [trusting_galois] DSL2 - revision: 1a2b3c4d5e
executor >  local (2)
[8f/2a1b3c] FASTQC (S1)      | 1/2 | 0/2 | 0/2 | 0% ✔
[7d/9e4f5a] FASTQC (S2)      | 1/2 | 0/2 | 0/2 | 0% ✔
```

If you see the two hash-prefixed directories and the sample tags in parentheses, the process conversion worked. Find the actual outputs in `work/8f/2a1b3c.../` and confirm the `*_fastqc.zip` files are there and open correctly.

### Step 6. Wire the Remaining Processes

Extend the module files with HISAT2 and SAMtools, then make the full workflow:

```groovy
process HISAT2_ALIGN {
    tag "${meta.id}"
    publishDir "${params.outdir}/aligned", mode: 'copy', pattern: "*.bam"

    input:
    tuple val(meta), path(reads)

    output:
    tuple val(meta), path("${meta.id}.bam"), emit: bam

    script:
    """
    hisat2 -x ${params.index} \\
        -1 ${reads} -2 ${reads[9]} \\
        -S ${meta.id}.sam \\
        2> ${meta.id}.hisat2.log
    samtools sort -@ ${task.cpus} -o ${meta.id}.bam ${meta.id}.sam
    samtools index ${meta.id}.bam
    rm ${meta.id}.sam
    """
}

process MULTIQC {
    publishDir "${params.outdir}/multiqc", mode: 'copy'

    input:
    path(fastqc_zips, stageAs: 'fastqc/*')

    output:
    path "multiqc_report.html"

    script:
    """
    multiqc fastqc
    """
}

workflow {
    ch_samples = Channel.fromPath(params.samplesheet, checkIfExists: true)
        .splitCsv(header: true, sep: '\t')
        .map { row -> tuple([id: row.sample], file(row.fq1), file(row.fq2)) }
        .map { meta, r1, r2 -> tuple(meta, [r1, r2]) }

    ch_trimmed = CUTADAPT(ch_samples)
    ch_bams    = HISAT2_ALIGN(ch_trimmed).bam
    ch_bams.view { meta, bam -> "Aligned ${meta.id} -> ${bam}" }
}
```

Note two structural differences from the bash script. The HISAT2 process bundles sorting and indexing into one process because they always run together on the same file, and there is a final collecting step in which many samples flow into one MultiQC report. Collecting patterns (`ch.collect()`) versus scattering (`ch.splitFastq`) is where Nextflow rewards a little study of the channel operators in the official documentation.

If your pipeline includes Python or R helper steps, leave them as-is inside the process script blocks. Nextflow does not care what language the body uses, and many production pipelines embed short Python [preprocessing](/blog/data-analysis/what-is-data-preprocessing) steps rather than rewriting them. The conversion goal is orchestration, not language purity.

### Step 7. Write `nextflow.config` With Containers and Resources

```groovy
params {
    samplesheet = "samples.csv"
    outdir      = "results"
    index       = "reference/grch38_index"
}

docker.enabled = true

profiles {
    test {
        params.samplesheet = "data/test_samples.csv"
        docker.enabled = true
    }
    singularity {
        docker.enabled = false
        singularity.enabled = true
        singularity.autoMounts = true
    }
    slurm {
        process.executor = 'slurm'
        singularity.enabled = true
    }
}

process {
    withLabel: 'process_medium' {
        cpus   = 4
        memory = 16.GB
        time   = 4.h
    }
}

process FASTQC {
    container 'quay.io/biocontainers/fastqc:0.12.1--hdfd78af_0'
}
process CUTADAPT {
    container 'quay.io/biocontainers/cutadapt:4.9--py312h2ec4fdc_1'
    label 'process_medium'
}
process HISAT2_ALIGN {
    container 'quay.io/biocontainers/hisat2:2.2.1--h6cc9453_6'
    label 'process_medium'
}
process MULTIQC {
    container 'quay.io/biocontainers/multiqc:1.25.1--pyhdfd78af_0'
}
```

The container tags above follow the BioContainers naming convention used across nf-core pipelines. Pull the tags that match the versions your bash script actually used, which you can get with `fastqc, version` and equivalents on the machine where the script last ran. Recording those versions is part of the conversion. If you skip it, the first full rerun under containers may produce slightly different numbers than the bash era, and you will not be able to say why.

### Step 8. Run the Full Pipeline and Read the Output

```bash
nextflow run main.nf -profile test
```

Expected output for a two-sample test sheet:

```
executor >  local (9)
[1a/2b3c4d] FASTQC (S1)       [100%] 2/2 ✔
[5e/6f7a8b] CUTADAPT (S1)     [100%] 2/2 ✔
[9b/0c1d2e] HISAT2_ALIGN (S1) [100%] 2/2 ✔
[3c/4d5e6f] FASTQC (S2)       [100%] 2/2 ✔
[7g/8h9i0j] CUTADAPT (S2)     [100%] 2/2 ✔
[1k/2l3m4n] HISAT2_ALIGN (S2) [100%] 2/2 ✔
[5o/6p7q8r] MULTIQC           [100%] 1/1 ✔
Aligned S1 -> S1.bam
Aligned S2 -> S2.bam
```

Then check the deliverables:

```bash
ls results/aligned/
```

```
S1.bam  S1.bam.bai  S2.bam  S2.bam.bai
```

And validate one result against the bash era:

```bash
samtools flagstat results/aligned/S1.bam
```

For the conversion to be called successful, the mapping statistics from the Nextflow run should match what the bash script produced on the same input, given the same tool versions. Small differences usually trace to tool version drift between your old environment and the new containers. This comparison, where you tabulate per-sample metrics side by side, is the same discipline that makes results tables legible in published work, where well-constructed tables carry much of the evidentiary weight of a study [8].

### Step 9. Learn the Resume Feature

Kill a run midway (Ctrl-C), fix nothing, and rerun with:

```bash
nextflow run main.nf -profile test -resume
```

Expected output includes lines like:

```
[1a/2b3c4d] FASTQC (S1)       [cached] 2/2
```

Every completed task whose inputs, script, container, and resources are unchanged is marked `[cached]` and skipped. This is the feature that justifies the whole exercise. On a 200-sample run where step three failed on six samples, resume turns a two-day rerun into a two-minute one. It also makes iterative development sane, because you can touch one process and recompute only the affected subgraph.

### Step 10. Put the Pipeline Under Version Control and Add a Test Profile

Initialize git, commit `main.nf`, the `modules/` directory, `nextflow.config`, and a CI-friendly test invocation such as:

```bash
nextflow run main.nf -profile test -stub-run
```

`-stub-run` executes each process as a stub that only creates its declared outputs, which validates the channel plumbing and the DAG in seconds without running the real tools. Full-scale testing on real data, even a minimal two-sample slice, belongs in the same repository. Teams that skip the test profile discover breakage only during a production run, at the worst possible moment.

## Key Directives and Options at a Glance

| Directive or option | What it does | Typical value in a conversion |
|--|--|--|
| `publishDir` | Copies process outputs out of `work/` into a stable results folder | `"${params.outdir}/fastqc", mode: 'copy'` |
| `container` | Pins the exact tool image for the process | `quay.io/biocontainers/fastqc:0.12.1` |
| `tag` | Human-readable label shown in logs and the DAG | `"${meta.id}"` |
| `label` | Attaches a resource group defined in config | `'process_medium'` |
| `cpus`, `memory`, `time` | Resource requests passed to the executor | `4`, `16.GB`, `4.h` |
| `errorStrategy` | What happens on failure | `'retry'` for transient cluster faults |
| `checkIfExists` | Fails fast when an input file is missing | `true` on all `fromPath` calls |
| `-resume` | Reuses cached tasks from the last run | Used on every rerun |
| `-stub-run` | Validates the DAG without executing tools | CI and smoke tests |
| `-profile` | Selects a config block (docker, slurm, test) | `-profile slurm` on the cluster |

## Convert Bash Script to Nextflow Tips

These are the tips that separate a conversion that works from one that people actually keep using.

**Keep each process single-purpose.** The bash script bundled trimming and QC per sample, but QC is per-read-file and trimming is per-pair. In Nextflow, mirror the tool boundaries, not the bash line boundaries. A process that runs three tools in sequence is fine when they always travel together (sort and index), and a smell when they are logically separable.

**Parameterize everything, hard-code nothing.** Every path, index, and threshold in your bash script becomes a `params` entry with a default in `nextflow.config`. Anyone rerunning your pipeline six months later should be able to change behavior with flags and a config file alone.

**Use nf-core module patterns even if you never join the community.** The nf-core project maintains community-curated, containerized Nextflow modules with standardized inputs, outputs, and version reporting. Copying their structure (meta maps, subworkflows, `modules.config`) buys you years of debugged conventions for free, and their pipelines serve as the de facto reference implementation of the patterns this guide describes.

**Adopt a small validation step inside the pipeline.** A final process that checks sample counts, runs `samtools quickcheck` on every BAM, and fails loudly if anything is off turns silent corruption into an immediate, located error. In data-intensive fields, explicit checkpoints across the analysis life cycle are increasingly treated as a requirement rather than a nicety, because they catch selection and processing problems before they propagate to interpretation [4].

**Teach it, do not just push it.** The single strongest predictor of whether a lab actually adopts the converted pipeline is whether someone walks the group through it. Hands-on workshop formats, in which participants run a real tool end to end during the session, measurably improve post-training performance over lecture-only formats [9]. One 60-minute session where each labmate runs the test profile themselves is worth more than a page of documentation.

**Keep the bash script in the repository, marked legacy.** Until the Nextflow version has produced results for a full project cycle, the bash script is your ground truth for behavior. Delete it only when you would genuinely never consult it again.

## Common Mistakes and Limitations

![Decision tree from a failed task to four causes: unescaped dollar sign, missing container, missing publishDir, oversized resource request.](/article-images/illustrations/convert-bash-script-to-nextflow-first-conversion-failures.webp)

These are the errors that show up in nearly every first conversion, with the exact messages and fixes.

**1. Unescaped dollar signs in the script block.** Symptom: bash reports `command not found` or an empty filename where a variable should be, and the error text shows a literal `$` that you expected to be substituted. Cause: Groovy interpolated the value before bash ran, or failed to interpolate a bash-only variable. Fix: use Nextflow variables for anything Nextflow knows (`${meta.id}`, `${reads}`) unescaped, and escape literal bash variables as `\$var`. If a block of bash is interpolation-heavy, move it to a template file with `template` and interpolate only the few values you need.

**2. Running with the wrong container and blaming the pipeline.** Symptom: `Process 'FASTQC' terminated with an error exit status (127)`, meaning command not found. Cause: the process has no container assigned, or `docker.enabled` is false while the local environment lacks the tool. Fix: assign a container in `nextflow.config` and enable a container engine profile. Exit code 127 on a first Nextflow run is almost never a Nextflow bug.

**3. Assuming task directories are results.** Symptom: the pipeline finishes and `results/` is empty or partial. Cause: missing `publishDir` directives. Nextflow deliberately hides outputs in `work/<hash>/`. Fix: add `publishDir` to every process whose outputs a human needs, and understand that `work/` is a cache, not a deliverable, and should not be committed or shared.

**4. Resource requests that exceed the machine.** Symptom: `Process requirement exceeds available CPUs, req: 16 cores; avail: 8 cores` and immediate task failure on a local run. Cause: config defaults sized for the cluster. Fix: keep resource profiles modest for local and test runs, and size per-label requests for the real executor. This is also the most common reason a pipeline runs fine in a test profile and dies on day one of production.

**5. Channel topology errors.** Symptom: `Channel right operation ... invoked with incompatible types` or a process that receives fewer items than expected. Cause: mismatched tuple shapes. If FASTQC emits `tuple(val, path)` and your downstream process declares `tuple(val, path, path)`, the channel stalls or crashes. Fix: print the channel with `.view()` at the junction, compare the tuple shape to the process declaration, and fix one of the two.

**6. Forgetting `checkIfExists`.** Symptom: the pipeline "runs" and later steps fail mysteriously or produce empty outputs because a glob matched nothing. Fix: set `checkIfExists: true` on every `fromPath` and `file` call so Nextflow fails at launch with a message naming the missing file.

**7. Container version drift.** Symptom: mapping rates or QC metrics differ between the bash era and the Nextflow era on identical input. Cause: the container tags in config are newer than the versions the bash script used. Fix: recover the old versions, pin matching tags, and record the discrepancy in the project notes. A conversion that changes scientific results is a failed conversion even if it runs green.

**Limitations of the conversion itself.** A Nextflow pipeline does not fix a scientifically weak bash script. It restarts faster, scales better, and audits cleanly, but it re-executes whatever logic you wrote, including the bugs. Individual analyses also still need expert judgment about parameters, reference builds, and interpretation, and for clinical or veterinary diagnostic work that judgment belongs with a qualified professional reviewing each case. Finally, the conversion effort scales with pipeline complexity. A 40-line script is an afternoon. A 2,000-line script with conditional branching, emailing, and database writes may be a multi-week project, and for such cases adopting an existing nf-core pipeline and migrating only your custom steps is frequently the cheaper path.

## How to Check the Result Is Right

Run this checklist after every meaningful change, not just once.

1. **Determinism check.** Run the pipeline twice on the test data with `-resume` in between and confirm the second run reports everything `[cached]`.
2. **Equivalence check.** On one sample, compare per-step outputs between the bash script and the pipeline (FastQC per-base quality plots, Cutadapt trimmed length distributions, `samtools flagstat` totals). They should agree within tool-version noise, and you should know the tool versions of both.
3. **Failure injection.** Corrupt one FASTQ deliberately and confirm the pipeline fails on that sample, reports which one via the `tag` directive, and that `-resume` skips only completed work on rerun.
4. **Trace audit.** After a run, inspect `pipeline_trace.txt` (enabled with `trace.enabled = true` in config) for per-task exit status, CPU time, memory, and container image. This file is what you attach to a methods section or internal SOP.
5. **Fresh-machine test.** Clone the repository on a second machine or container-only environment and run the test profile with no local tool installations. If it passes, the pipeline is genuinely portable, which is the property that separates a workflow from a well-organized script.

## Next Steps

Once the pipeline is stable, three upgrades deliver the most value in order. First, adopt `nf-core/tools` module structure so individual processes become shareable modules. Second, add a `CI.yml` GitHub Actions workflow that runs `-stub-run` plus the test profile on every pull request. Third, survey the nf-core pipeline collection for a community pipeline close to what you built, because migrating your custom steps into a maintained, peer-reviewed pipeline is usually more durable than maintaining your own long term. That migration, like the original conversion, is a structured practice change, and the implementation literature is unambiguous that planned rollouts with training and an enabling environment outperform big-bang rewrites.

## Frequently Asked Questions

### Do I have to rewrite my whole bash script, or can Nextflow call it directly?

You can call an existing bash script from a process, and that is a legitimate first migration step. Wrap the whole script in one process, containerize it, and refactor into separate processes gradually. A pipeline that is one giant process is valid Nextflow but gives up parallelism per step and granular resume, so treat it as a waypoint, not a destination.

### What version of Java and Nextflow do I need?

Install Java 17 or later and any recent Nextflow release from the last two years. Recent releases default to DSL2, which is the syntax this guide uses, so no special flag is needed. Check with `nextflow -v` after installation.

### How does Nextflow compare to Snakemake for someone who knows bash but not Python?

Nextflow's DSL is Groovy-based, but day-to-day pipeline writing requires little Groovy beyond interpolation and channel operators. Snakemake requires comfort with Python idioms. If your only language is bash, Nextflow is usually the gentler entry, and its nf-core module ecosystem gives you copyable patterns immediately.

### Can Nextflow run my pipeline on an HPC cluster without changing the pipeline code?

Yes. Add a profile in `nextflow.config` that sets `process.executor = 'slurm'` (or `sge`, `pbs`, `lsf` as appropriate) and enables Singularity or Apptainer. The pipeline code stays identical, which is one of the strongest reasons to convert in the first place.

### Why does Nextflow put everything in a `work` directory instead of my results folder?

Every task runs in an isolated, hash-named directory so that Nextflow can detect exactly what changed between runs and cache the rest. Outputs reach your project only through `publishDir`. Treat `work/` as a cache you can delete, and treat published results as the deliverables.

### What does the `-resume` flag actually reuse?

Resume reuses any task whose inputs, script text, container, and resource requests are unchanged since the last successful run, based on content hashing. If you edit one process, only that process and everything downstream reruns. Unaffected tasks print as `[cached]` and complete in seconds.

### How do I handle paired-end reads so R1 and R2 never get mismatched?

Carry the pair through the pipeline as a single tuple, `tuple val(meta), path(reads)`, with both files staged together, and derive output filenames from the sample ID in `meta` rather than from the FASTQ filenames. A sample sheet validated at channel construction time, with `checkIfExists: true`, catches missing or duplicated files at launch instead of at alignment.

### Is it safe to keep running my bash script while I convert?

Yes, and you should. Keep the bash script as the reference implementation while you build and validate the Nextflow version on test data, and switch over only after an equivalence check confirms both produce the same results. Run them in separate project directories so results cannot be confused.

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [
    {
      "@type": "Question",
      "name": "Do I have to rewrite my whole bash script, or can Nextflow call it directly?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "You can call an existing bash script from a single Nextflow process as a first migration step, then refactor into separate processes over time. A one-process pipeline works but gives up per-step parallelism and granular resume, so treat it as a waypoint."
      }
    },
    {
      "@type": "Question",
      "name": "What version of Java and Nextflow do I need?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Install Java 17 or later and a recent Nextflow release. Releases from the last two years default to DSL2 syntax, so no special flag is needed. Verify the install with nextflow -v."
      }
    },
    {
      "@type": "Question",
      "name": "How does Nextflow compare to Snakemake for someone who knows bash but not Python?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Both are excellent workflow managers. Nextflow uses a Groovy-based DSL that requires little Groovy day to day, while Snakemake requires comfort with Python idioms. Bash-only users generally find Nextflow the gentler entry, helped by the nf-core module ecosystem."
      }
    },
    {
      "@type": "Question",
      "name": "Can Nextflow run my pipeline on an HPC cluster without changing the pipeline code?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Yes. Add a config profile that sets process.executor to slurm, sge, pbs, or lsf and enables Singularity or Apptainer. The pipeline code itself stays identical across laptop, cluster, and cloud."
      }
    },
    {
      "@type": "Question",
      "name": "Why does Nextflow put everything in a work directory instead of my results folder?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Each task runs in an isolated hash-named directory so Nextflow can detect what changed between runs and cache the rest. Outputs reach your project only through the publishDir directive. Treat work as a cache and published results as the deliverables."
      }
    },
    {
      "@type": "Question",
      "name": "What does the -resume flag actually reuse?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Resume reuses any task whose inputs, script text, container, and resource requests are unchanged since the last successful run, using content hashing. Edited processes and everything downstream rerun, while unaffected tasks print as cached and complete in seconds."
      }
    },
    {
      "@type": "Question",
      "name": "How do I handle paired-end reads so R1 and R2 never get mismatched?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Carry the pair as a single tuple with both files staged together, derive filenames from the sample ID in the meta map rather than from FASTQ filenames, and validate the sample sheet at channel construction with checkIfExists set to true."
      }
    },
    {
      "@type": "Question",
      "name": "Is it safe to keep running my bash script while I convert?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Yes. Keep the bash script as the reference implementation while validating the Nextflow version on test data, and switch over only after an equivalence check confirms both produce the same results. Run them in separate project directories."
      }
    }
  ]
}
</script>

## Sources

1. [Getting Started in Probabilistic Graphical Models](https://pubmed.ncbi.nlm.nih.gov/18069887/)
2. [Getting Started on Artificial Intelligence in Health Care and Clinical Research: Includes Rigor Checklist for Authors and Reviewers](https://pubmed.ncbi.nlm.nih.gov/41039915/)
3. [Getting Small Medium Enterprises started on Industry 4.0 using retrofitting solutions](https://doi.org/10.1016/j.promfg.2020.04.096)
4. [Ten Core Concepts for Ensuring Data Equity in Public Health.](https://pubmed.ncbi.nlm.nih.gov/41511794/)
5. [Concept Mapping in Magnetism and Electrostatics: Core Concepts and Development over Time](https://doi.org/10.3390/educsci10050129)
6. [Getting Started with Functional Patterns](https://doi.org/10.1007/978-1-4842-2958-3_3)
7. [Data Analysis and Machine Learning: Getting started, our first data and Machine Learning encounters](https://www.semanticscholar.org/paper/2610c80715d36183019cc88266cd847b1f6bfbe2)
8. [Getting started with tables](https://pubmed.ncbi.nlm.nih.gov/28321295/)
9. [Getting Started with Machine Learning in Game Development](https://doi.org/10.55824/jpm.v4i2.539)