# Mastering Snakemake Wildcards: How to Write Flexible Rules By the end of this guide you will be able to write Snakemake rules that process one sample or a thousand without editing the workflow, diagnose every common wildcard error from its exact message, and organize a project the way published pipelines in population genomics, metagenomics, and RNA-seq do. You need a Unix-like shell, Python 3.11 or newer (which recent Snakemake releases require), roughly 20 MB of free disk space, and an internet connection for the initial install. No cluster, root access, or prior workflow experience is required. Snakemake is a Python-based bioinformatics workflow manager first published by Köster and Rahmann in 2012 [1]. Its core idea is the wildcard, a named variable embedded in file patterns that lets a single rule describe work for every sample, lane, or contig in a study. The original publication introduced Snakemake as the first workflow system to support automatically inferred multiple named wildcards in input and output filenames [1]. Mastering Snakemake wildcards is therefore the highest-leverage skill for turning a hardcoded script into a flexible pipeline, and it is the subject of this guide. ## What a Wildcard Is (and What It Is Not) A wildcard is a named placeholder in braces that appears in a rule's input or output file pattern. In the output pattern `results/trimmed/{sample}.fastq.gz`, the part `{sample}` is the wildcard. When you ask Snakemake to produce `results/trimmed/sample_A.fastq.gz`, it matches the pattern against the requested file name, infers `sample=sample_A`, and substitutes that value everywhere the same wildcard name appears in the rule. The 2012 paper describes this as automatic inference of wildcard values from file names [1]. Two clarifications prevent most early confusion. First, Snakemake wildcards are not shell wildcards. A shell wildcard such as `*.fastq.gz` means "any characters", while a Snakemake wildcard is a named variable whose value the scheduler tracks. Second, a wildcard value comes only from a file name. Snakemake never opens a file to decide what `{sample}` means, and the complete job graph is computed before anything runs. Both properties follow from the original design goal, a workflow definition that scales from a single workstation to a compute cluster without modification [1]. ## Why Wildcards Beat Shell Loops A common first attempt at multi-sample processing is a shell loop: ```bash for f in data/raw/*.fastq.gz do cutadapt -a AACCGGTT -o results/trimmed/$(basename $f) $f done ``` This works once and then causes pain. The loop reruns every sample on every execution, it cannot skip work that is already done, it offers no parallel scheduling across samples, and a failure halfway leaves no record of what completed. A wildcard rule plus a target list gives you all of that without writing loop logic. Snakemake builds a directed acyclic graph (DAG), the map of jobs and file dependencies it computes from your rules, then runs only the jobs needed for the requested outputs, in parallel where possible, and skips anything already up to date [1]. ## Snakemake Wildcards Step by Step: A Working Mini-Pipeline This section builds a small but complete pipeline for three samples. Every command is real, and every output block shows what you should see. Exact timestamps, job ids, and line ordering vary between Snakemake versions. ### Step 1: Install Snakemake, cutadapt, and MultiQC ```bash python3 --version # Python 3.12.4 (you need 3.11 or newer) python3 -m venv .venv source .venv/bin/activate pip install snakemake cutadapt multiqc pandas snakemake --version # 9.x.x (any Snakemake 8 or 9 release works for this guide) ``` cutadapt removes adapter sequences from FASTQ files (the standard format for raw sequencing reads) and MultiQC aggregates per-sample tool logs into one HTML report. Both install with pip, which keeps this tutorial free of environment managers. Published Snakemake workflows typically pin tools with Conda environments or containers instead, a pattern covered in the tips section [2][3]. ### Step 2: Create the Project and Fake Data ```bash mkdir -p config data/raw logs/cutadapt results printf "sample\nsample_A\nsample_B\nsample_C\n" > config/samples.tsv printf 'adapter: "AACCGGTT"\n' > config/config.yaml ``` Now generate synthetic reads: ```bash python3 - <<'EOF' import gzip, random random.seed(42) for s in ["sample_A", "sample_B", "sample_C"]: with gzip.open(f"data/raw/{s}.fastq.gz", "wt") as fh: for i in range(100): seq = "".join(random.choices("ACGT", k=80)) + "AACCGGTT" qual = "I" * len(seq) fh.write(f"@{s}_read{i}\n{seq}\n+\n{qual}\n") EOF ``` Each file holds 100 reads of 88 bases ending in the 8-base adapter `AACCGGTT`, so cutadapt has real work to do. The fixed random seed makes the data reproducible. ### Step 3: Write the Snakefile Create a file named `Snakefile` in the project root with this content: ```python import pandas as pd configfile: "config/config.yaml" samples = pd.read_csv("config/samples.tsv", sep="\t", dtype=str) SAMPLES = samples["sample"].tolist() rule all: input: expand("results/trimmed/{sample}.fastq.gz", sample=SAMPLES), "results/multiqc_report.html" rule cutadapt: input: "data/raw/{sample}.fastq.gz" output: "results/trimmed/{sample}.fastq.gz" log: "logs/cutadapt/{sample}.log" params: adapter=config["adapter"] shell: "cutadapt -a {params.adapter} -o {output} {input} 2> {log}" rule multiqc: input: expand("logs/cutadapt/{sample}.log", sample=SAMPLES) output: "results/multiqc_report.html" log: "logs/multiqc.log" shell: "multiqc {input} -o results -n multiqc_report.html 2> {log}" ``` Three elements carry the wildcard design. The `cutadapt` rule is written once with `{sample}` in its input, output, and log paths. `rule all` uses `expand()` to turn that pattern into one concrete target per sample. The `multiqc` rule has no wildcards of its own, it consumes the per-sample logs and aggregates them. The pandas line reads the sample sheet, which is the single source of truth for which samples exist. ### Step 4: Dry-Run to Inspect the Plan ```bash snakemake -n -p ``` `-n` means dry run, Snakemake computes the DAG and prints the plan without executing. `-p` prints each shell command during a real run. Expected output (abridged): ``` Building DAG of jobs... Job stats: job count --------- ----- all 1 cutadapt 3 multiqc 1 total 5 This is a dry-run, no output files will be created. ``` The table says three `cutadapt` jobs, one per wildcard value, plus one `multiqc` job and the bookkeeping rule `all`. If this table looks wrong, fix the Snakefile now, before any compute is spent. ### Step 5: Draw the DAG ```bash snakemake --dag | dot -Tsvg > dag.svg ``` This needs Graphviz (install it with your package manager, for example `apt install graphviz`). The graph shows three parallel cutadapt branches feeding one multiqc node, the shape you will see in every wildcard-driven pipeline. ### Step 6: Run It ```bash snakemake -j 2 -p ``` `-j 2` sets the number of parallel cores. Expected output (abridged): ``` Building DAG of jobs... [Tue Nov 11 09:14:01 2026] rule cutadapt: input: data/raw/sample_A.fastq.gz output: results/trimmed/sample_A.fastq.gz log: logs/cutadapt/sample_A.log jobid: 2 reason: Missing output files: results/trimmed/sample_A.fastq.gz wildcards: sample=sample_A [Tue Nov 11 09:14:02 2026] Finished job 2. 1 of 5 steps (20%) done ... 5 of 5 steps (100%) done ``` The `wildcards:` line confirms the inferred value for each job. With `-j 2`, two cutadapt jobs run at once. ### Step 7: Verify the Outputs ```bash zcat results/trimmed/sample_A.fastq.gz | head -2 # @sample_A_read0 # (80 random bases, the adapter is gone) tail -n 6 logs/cutadapt/sample_A.log ``` Expected in the log (abridged): ``` Total reads processed: 100 Reads written (pass 1): 100 (100.0%) Total basepairs processed: 8,800 bp Reads with adapters: 100 (100.0%) ``` Every read carried the adapter, so cutadapt trimmed all 100 reads and each lost its 8-base tail. Open `results/multiqc_report.html` in a browser. You should see one cutadapt panel per sample, because each log file carried a distinct wildcard value. ### Step 8: Add a Sample and Rerun ```bash printf "sample_D\n" >> config/samples.tsv python3 - <<'EOF' import gzip, random random.seed(43) with gzip.open("data/raw/sample_D.fastq.gz", "wt") as fh: for i in range(100): seq = "".join(random.choices("ACGT", k=80)) + "AACCGGTT" qual = "I" * len(seq) fh.write(f"@sample_D_read{i}\n{seq}\n+\n{qual}\n") EOF snakemake -n ``` Expected: ``` Job stats: job count --------- ----- all 1 cutadapt 1 multiqc 1 total 3 ``` Only `sample_D` gets a cutadapt job. The three finished samples are up to date and skipped, and multiqc reruns because its input list changed. This is the payoff of wildcards plus a target list, incremental processing with no loop logic written by you. Execute the plan with `snakemake -j 2`. ## Wildcard Syntax at a Glance | Construct | Where it appears | What it does | |, - |, - |, - | | `{sample}` | output pattern | Declares the wildcard and captures its value from a requested target file name | | `{sample}` | input pattern | Substitutes the captured value to locate inputs | | `{wildcards.sample}` | shell string | Inserts the captured value into the command (the bare `{sample}` also works) | | `wildcards.sample` | run, script, and params functions | Exposes the value as a Python string | | `expand(pattern, sample=SAMPLES)` | target lists, usually rule all | Expands a pattern into concrete file names | | `wildcard_constraints: sample="..."` | top level | Restricts a wildcard with a regular expression to resolve ambiguity | | `{sample,\d+}` | single output pattern | Inline constraint for one pattern | | checkpoints with input functions | rule level | Discovers new wildcard values after a job runs | ## How Inference and Backtracking Work Wildcards become powerful when rules chain. Add a counting stage to the mini-pipeline: ```python rule count_reads: input: "results/trimmed/{sample}.fastq.gz" output: "results/counts/{sample}.tsv" shell: "zcat {input} | awk 'END {{print NR/4}}' > {output}" ``` When you request `results/counts/sample_A.tsv` (from the command line or through `rule all`), Snakemake matches the `count_reads` output pattern, infers `sample=sample_A`, formats the input as `results/trimmed/sample_A.fastq.gz`, discovers that no such file exists, matches the `cutadapt` rule with the same sample value, and continues until it reaches `data/raw/sample_A.fastq.gz`, which exists. This backward walk is called backtracking. The same wildcard name is the thread that ties the chain together. You can also request a single concrete file directly, for example `snakemake -n results/trimmed/sample_B.fastq.gz`, and Snakemake plans only that branch. Three properties follow from this design. Wildcard values are plain strings inferred from file names. The DAG is fixed before execution, so rules cannot discover new work mid-run unless you use checkpoints. And a wildcard that appears in an input pattern but not in the output pattern of the same rule is always an error, because nothing determines its value. If several rules could produce the same file, Snakemake refuses to guess and raises an exception, covered in the constraints section below. ## Multiple Wildcards: Paired-End Reads, Lanes, and Units Paired-end data, where each fragment is sequenced from both ends and stored as two mate files, needs one wildcard for the sample and two named file slots: ```python rule cutadapt_pe: input: r1="data/raw/{sample}_R1.fastq.gz", r2="data/raw/{sample}_R2.fastq.gz" output: r1="results/trimmed/{sample}_R1.fastq.gz", r2="results/trimmed/{sample}_R2.fastq.gz" log: "logs/cutadapt/{sample}.log" params: a1="AACCGGTT", a2="TTGGCCAA" shell: "cutadapt -a {params.a1} -A {params.a2} " "-o {output.r1} -p {output.r2} " "{input.r1} {input.r2} 2> {log}" ``` Named input and output entries let the shell command refer to `{input.r1}` and `{output.r2}` explicitly, which prevents mate swapping, a classic silent error. One inferred `{sample}` value fills all four file patterns. Two wildcards also combine in one path, for example `results/trimmed/{sample}/{unit}.fastq.gz`, where `unit` distinguishes sequencing lanes or library fractions. Published pipelines lean on this heavily. DamMapper processes multiple biological replicates per condition and visualizes quality control across them with correlation heatmaps and [principal component analysis](/blog/data-analysis/principal-component-analysis-explained) [2]. CapMux resolves pooled single-cell data into sample-resolved outputs by combining sub-library index information with the experimental layout, exactly the kind of two-dimensional wildcard space (sample by sub-library) that path-based wildcards express [4]. Beware collisions when patterns overlap. A single-end rule with output `results/trimmed/{sample}.fastq.gz` and a paired-end rule with output `results/trimmed/{sample}_R1.fastq.gz` both match the file `results/trimmed/sample_A_R1.fastq.gz`, the first with `sample=sample_A_R1`. The cleanest fix is structural, put each data type in its own directory (`results/trimmed/se/` and `results/trimmed/pe/`). The regex-based fix is covered next. ## expand(): Turning Patterns into Target Lists ![Side-by-side comparison of expand() product form and union form with their inputs and outputs.](/article-images/illustrations/snakemake-wildcards-expand-forms.webp) `expand()` is Snakemake's function for applying one or more wildcard patterns to concrete value lists. It comes in two forms. Product form, one output for every combination: ```python expand("results/trimmed/{sample}/{unit}.fastq.gz", sample=["sample_A", "sample_B"], unit=["lane1", "lane2"]) ``` returns ```python ['results/trimmed/sample_A/lane1.fastq.gz', 'results/trimmed/sample_A/lane2.fastq.gz', 'results/trimmed/sample_B/lane1.fastq.gz', 'results/trimmed/sample_B/lane2.fastq.gz'] ``` Union form, several patterns for the same values: ```python expand(["results/trimmed/{sample}.fastq.gz", "results/counts/{sample}.tsv"], sample=SAMPLES) ``` returns the trimmed file and the count table for every sample in one list. Union form is what you want in `rule all` when a stage produces several outputs per sample. Two habits keep `expand()` safe. Build value lists from the sample sheet, never from `glob()` scans of the results directory, so the DAG depends only on declared inputs. And watch the combinatorics, a product of 200 samples by 10 units by 3 method variants is 6,000 jobs, which Snakemake handles but which you should see coming. ## Wildcard Constraints: Resolving Ambiguity with Regular Expressions A wildcard matches any characters by default, including slashes and dots. That power creates ambiguity when two rules can produce the same file. The classic case uses two patterns that differ only by an extra wildcard: ```python rule sample_level: output: "results/{dataset}.txt" rule group_level: output: "results/{dataset}.{group}.txt" ``` The target `results/1.control.txt` matches both rules, `sample_level` with `dataset=1.control`, and `group_level` with `dataset=1` and `group=control`. Snakemake raises: ``` AmbiguousRuleException in line 12 of /path/to/Snakefile: Rules sample_level and group_level are ambiguous for the production of results/1.control.txt. ``` The fix is a wildcard constraint, a regular expression (a pattern language for matching text) that restricts what a wildcard value may contain: ```python wildcard_constraints: dataset="\d+" ``` Now `dataset` may only contain digits, so `sample_level` cannot match `results/1.control.txt` and the ambiguity disappears. Constraints are global by wildcard name, they apply in every rule that uses it. For one-off cases you can also write the constraint inline in a pattern, for example `results/trimmed/{sample,[A-Za-z0-9_]+}.fastq.gz`. Constraints also stop wildcards from swallowing directory separators. A pattern like `results/{path}/report.html` given the file `results/a/b/report.html` infers `path=a/b`, which is occasionally what you want and usually not. Constrain `path="[^/]+"` if a single directory level is intended. ## Sample Sheets and Config Files: Where Wildcard Values Should Come From The step-by-step pipeline already shows the recommended split. Wildcard values (which samples exist) come from a sample sheet read with pandas. Parameters (which adapter, which thresholds) come from a YAML config loaded with the `configfile` directive and accessed through the `config` object. This separation is standard practice in published workflows. Tourmaline, a containerized amplicon analysis workflow built on QIIME 2 and Snakemake, starts from a configuration file that defines parameters and input files, so the same Snakefile serves many analyses [5]. epiGBS2 similarly exposes flexible parameter settings through its workflow configuration [6]. Keep the sample sheet minimal. A tab-separated file with a `sample` column and optional `unit` and `condition` columns covers most designs. Validate early, because a sample listed in the sheet without a raw file produces this error at job time: ``` MissingInputException in rule cutadapt (line 18 of Snakefile): Missing input files for rule cutadapt: data/raw/sample_E.fastq.gz ``` That message almost always means a typo in either the sheet or the file name. A short validation rule or script that checks the sheet against the filesystem at DAG time turns this into a clear, early failure instead. ## Wildcard Patterns in Published Workflows Reading published Snakefiles is the fastest way to absorb wildcard idioms. Three structures recur. **Per-sample fan-out with an aggregate.** Each sample flows through the same chain, then one aggregate rule collects everything into a report. PopGLen runs population genomic analyses across a dataset and compiles results into an HTML report [7]. gyōza takes deep-mutational scanning data from raw reads to fitness scores with quality control and an automatically generated HTML report [8]. Tourmaline does the same for amplicon analyses [5]. The aggregate rule holds no wildcards, its input is an `expand()` list. **Modular stage chains with stable wildcard interfaces.** Large pipelines split into stages that exchange files whose names follow one wildcard vocabulary. ATLAS takes metagenome data from raw reads through assembly, binning, and annotation as a customizable modular workflow [9]. VIPER moves RNA-seq data from raw reads through alignment to differential expression and pathway analysis in modular fashion [10]. HaploCharmer chains mapping, haplotype identification, filtration, and reporting for a panel of accessions [11]. TriosCompass integrates multiple variant caller types into one configurable framework for de novo variant discovery [12]. In each case, module boundaries fall where wildcard meanings stay stable. **Wildcards over more than samples.** Anything that varies per output can be a wildcard. CapMux indexes outputs by sample and sub-library [4]. Menzel's long-read bacterial workflows run several genome assembly and evaluation programs at once so users can compare results, a benchmarking pattern you can express with an `{assembler}` wildcard [13]. The museum-genomics toolkit of White and colleagues ships three pipelines (skim2mt, skim2rrna, and gene2phylo) designed to run as separate stages over shared inputs [14]. ## Snakemake Wildcards Tips from Published Pipelines 1. **Fix a canonical wildcard vocabulary.** Choose `sample`, `unit`, and `run` (or equivalents) once and use those exact names in every rule. Wildcard names are the join keys of your pipeline, inconsistent names break inference. 2. **Drive everything from the sample sheet and config.** If adding a sample requires editing the Snakefile, the design is wrong. Tourmaline's configuration-driven design is the model [5]. 3. **Keep aggregate rules wildcard-free.** A rule like `multiqc` should take `expand()` lists and produce one file. If an aggregate rule has wildcards, ask whether you actually want one aggregate per group, which is a deliberate `{group}` wildcard. 4. **Write `wildcard_constraints` before patterns overlap**, not after the first AmbiguousRuleException. 5. **Never retype file paths inside shell commands.** Always write `{input}`, `{output}`, and `{log}` placeholders. Retyped paths are the main cause of MissingOutputException, because the command writes a file that does not match the declared output. 6. **Escape literal braces in shell strings.** Snakemake formats the shell string with Python before execution, so awk and sed braces must double. Write `awk 'END {{print NR/4}}'` as in the counting rule above. Forgetting this raises a `KeyError` naming the text inside the braces. 7. **Use `temp()` for intermediates and `protected()` for finals.** `temp("results/trimmed/{sample}.fastq.gz")` deletes the file once every consumer has run, which keeps large cohorts from filling disks. `protected()` marks outputs that a rerun must not overwrite. 8. **Pin software per rule with conda or container directives.** DamMapper ensures reproducibility across HPC and cloud environments with Conda environments and Docker or Apptainer containers [2], and kGWASflow combines Snakemake with Conda and Docker to eliminate missing dependencies and version conflicts [3]. The directive form is one line per rule: ```python conda: "envs/cutadapt.yaml" ``` 9. **Give every wildcard value its own log path.** `logs/cutadapt/{sample}.log` turns a 500-sample failure into 500 independent debugging sessions instead of one interleaved mess. 10. **Treat method comparisons as wildcards, not copies.** Menzel built his assembly workflows because end users need to test many assembler combinations automatically [13]. One rule with an `{assembler}` wildcard expresses that, duplicated rules do not. ## Checkpoints: When Wildcard Values Are Unknown Until Runtime Ordinary wildcards require the complete DAG before anything runs. Checkpoints relax that. A checkpoint is a rule whose outputs may be inspected after it finishes, allowing downstream wildcard values to be discovered from the files it actually created (add `import os` at the top of the Snakefile): ```python checkpoint annotate_contigs: input: "results/assembly/{sample}.contigs.fa" output: directory("results/contigs/{sample}") shell: "python3 scripts/run_annotations.py {input} {output}" def contig_inputs(wildcards): ckpt = checkpoints.annotate_contigs(**wildcards).get(**wildcards) contigs = glob_wildcards( os.path.join(ckpt.output, "{contig}.fa")).contig return expand("results/contigs/{sample}/{contig}.gff", sample=wildcards.sample, contig=contigs) rule per_contig: input: contig_inputs output: "results/contigs/{sample}.merged.gff" shell: "cat {input} > {output}" ``` The input function `contig_inputs` runs after the checkpoint completes, globs the contig files the annotation step produced, and returns one target per contig. Snakemake then re-evaluates the DAG with the new wildcard values. Use checkpoints sparingly, they add re-evaluation complexity, and most per-sample designs never need them. ## Common Mistakes and Limitations ![Decision tree mapping four Snakemake wildcard error messages to their causes and fixes.](/article-images/illustrations/snakemake-wildcards-wildcard-error-triage.webp) Most failed first runs trace back to a handful of wildcard mistakes. Each one below includes the exact message and the fix. ### 1. Wildcards in the first rule The first rule in a Snakefile defines the default targets, and default targets must be concrete. A Snakefile whose first rule is a wildcard rule fails immediately: ``` Target rules may not contain wildcards. ``` Fix by defining `rule all` first, with explicit `expand()` targets, as the step-by-step pipeline does. ### 2. Inconsistent wildcard names An output of `results/counts/{sample}.tsv` with an input of `data/raw/{sample_id}.fastq.gz` produces: ``` WildcardError in line 9 of Snakefile: Wildcard 'sample_id' in input file 'data/raw/{sample_id}.fastq.gz' cannot be determined from output files. ``` (Wording varies slightly between Snakemake versions.) Fix by using one name everywhere, `{sample}` in every rule. ### 3. Overlapping output patterns Two rules that can produce the same file trigger `AmbiguousRuleException`, shown in the constraints section above. Fix with `wildcard_constraints` or with structurally distinct directories and suffixes. ### 4. Unescaped braces in shell strings A command like `awk '{print $1}'` inside a shell block raises: ``` KeyError: 'print $1' ``` Snakemake formats the shell string before execution, so every literal brace must double to `{{ }}`. This bites in awk, sed, perl, and `python3 -c` one-liners. ### 5. Command writes a different file than declared If the shell command writes `trimmed_{sample}.fq.gz` while the output block declares `results/trimmed/{sample}.fastq.gz`, the job succeeds and then: ``` MissingOutputException in rule cutadapt (line 20 of Snakefile): Job completed successfully, but some output files are missing. Missing output files: results/trimmed/sample_A.fastq.gz ``` Fix by using `{output}` placeholders exclusively in the command. ### 6. Sample sheet entries without files A listed sample with no raw file triggers `MissingInputException`, shown in the sample sheet section. Fix by validating the sheet against the filesystem before running. ### 7. Wildcards that swallow separators A wildcard matching across `/` creates unexpected nested paths and can make two patterns collide. Constrain with `[^/]+` or restructure directories. ### 8. Encoding parameters as wildcards A wildcard creates one parallel branch per value. Putting a k-mer size or a minimum length in a file name multiplies your DAG by every parameter value and reruns everything when you change it. Wildcards identify data, config parameters control behavior. The exception is a deliberate parameter sweep, where parallel branches are the goal. ### Limitations Wildcard matching is purely lexical. Two samples named `sample_A` and `sample_A.old` both match `{sample}` unless constrained, and wildcard values carry no type, order, or metadata. The static DAG means the job set is fixed before execution, checkpoints being the escape valve. Very large wildcard spaces (hundreds of thousands of jobs) increase DAG construction time and memory, which is one reason published pipelines modularize stages and keep per-branch fan-out bounded [8][12][10]. ## How to Verify a Wildcard-Driven Workflow Verification has three layers. **Plan check.** `snakemake -n` must show the job counts you expect, one job per sample for each per-sample rule and one aggregate. Any surprise in the count column points to a wildcard or `expand()` problem. **Wiring check without compute.** Dry-runs validate structure for free. If you generated files outside Snakemake (for example on a cluster), `snakemake, touch` stamps them so the workflow accepts them as existing outputs. **Result check.** After a real run, `snakemake -n` prints: ``` Nothing to be done (all requested files are present and up to date). ``` Then spot-check one sample end to end, read its log, inspect its output with `zcat` and `head`, and confirm the aggregate report contains one entry per sample. Snakemake records per-file provenance (which rule produced it, when, and against which workflow version) under `.snakemake/metadata`, and `snakemake, summary` lists every target with the rule that created it and its status. ## Nextflow vs Snakemake: How Wildcards Shape the Choice Both Snakemake and Nextflow are mature, widely deployed workflow managers, and both handle cohort-scale sequencing analysis well. The difference that matters for wildcard work is the parallelism model. Snakemake couples parallelism to file names. Wildcard patterns define the branches, `expand()` defines the targets, and the scheduler derives everything else. Per-sample structure is visible in the rule code, and incremental reruns are automatic. The cost is that your file naming scheme becomes an interface, so naming discipline is mandatory. Snakemake's Python-based definition language and its ability to scale from a workstation to a cluster without modifying the workflow were its founding claims [1], and published workflows now span metagenomics [9], RNA-seq [10], population genomics [7], long-read assembly [13], and single-cell processing [4]. Nextflow couples parallelism to channels, queues of data items that processes consume and emit, written in a Groovy-based language. Structure lives in channel wiring rather than file patterns, which some teams find more flexible for irregular data flows and others find less transparent than filename-driven logic. Practical selection criteria are straightforward. Pick the language your team will maintain (Python versus Groovy). Match the ecosystem you want to reuse. Weigh how much of your design is naturally per-sample, where wildcards excel, versus stream-like transformations. Both tools integrate Conda and containers, and both run on schedulers and cloud platforms. For wildcard-centric designs, Snakemake's model maps directly onto the rules you write. ## Where to Go Next 1. Add `conda` directives to each rule and run with the `use-conda` flag, following the reproducibility pattern of DamMapper and kGWASflow [2][3]. 2. Split the Snakefile into modules once it passes a few hundred lines, keeping one wildcard vocabulary across module boundaries. 3. Add a cluster or cloud execution profile. Scaling without workflow modification was a founding design goal of Snakemake [1]. 4. Read and run published workflows to absorb structure, including PopGLen [7], gyōza [8], and Menzel's assembly workflows [13], all distributed with documentation through public repositories. 5. Practice the dimensions: add a `{unit}` level to the mini-pipeline, add a QC-failure branch, then add a checkpoint that discovers contigs. ## Frequently Asked Questions ### What exactly is a Snakemake wildcard? A wildcard is a named variable in braces, such as `{sample}`, embedded in a rule's input or output file pattern. Snakemake infers its value by matching the pattern against concrete file names and substitutes that value throughout the rule. ### How do I run the same rule for many samples? Write the sample name as a wildcard in the rule and list concrete targets with `expand()` in `rule all`. Snakemake then creates and schedules one job per sample automatically. ### Why does Snakemake say target rules may not contain wildcards? Because the first rule in a Snakefile defines the default targets, and default targets must be concrete file names. Add a `rule all` with explicit `expand()` targets before your wildcard rules. ### What is the difference between expand() and a wildcard? A wildcard is a placeholder inside a single file pattern, while `expand()` applies one or more patterns to lists of values and returns concrete file names. You use `expand()` to build the target list that triggers wildcard inference. ### How do I handle paired-end reads with wildcards? Use one `{sample}` wildcard with two named input and output entries, commonly `r1` and `r2`. One inferred sample value then fills all four mate file patterns and keeps the mates paired. ### How do I fix an AmbiguousRuleException? Add a `wildcard_constraints` entry with a regular expression that pins the shared wildcard so only one rule can match each target file. Alternatively, make the patterns structurally distinct with different directories or suffixes. ### Can I use wildcards inside the shell command? Yes, reference them as `{wildcards.sample}` or the bare `{sample}` in the shell string. Escape any literal braces needed by awk or sed by doubling them to `{{ }}`. ### When do I need a checkpoint instead of ordinary wildcards? Use a checkpoint when the set of wildcard values, such as assembly contigs or clusters, is unknown until a job has run. Ordinary wildcards require the complete job graph to be known before execution. ## Sources 1. [Snakemake--a scalable bioinformatics workflow engine.](https://pubmed.ncbi.nlm.nih.gov/22908215/) 2. [Mapping SET1B chromatin interactions with DamID using DamMapper, a comprehensive Snakemake workflow](https://pubmed.ncbi.nlm.nih.gov/41087863/) 3. [kGWASflow: a modular, flexible, and reproducible Snakemake workflow for k-mers-based GWAS](https://pubmed.ncbi.nlm.nih.gov/37976215/) 4. [CapMux: a Snakemake pipeline for early demultiplexing of split-pool scRNA-seq data into sample-resolved outputs](https://pubmed.ncbi.nlm.nih.gov/42369763/) 5. [Tourmaline: A containerized workflow for rapid and iterable amplicon sequence analysis using QIIME 2 and Snakemake](https://pubmed.ncbi.nlm.nih.gov/35902092/) 6. [epiGBS2: an improved protocol and automated snakemake workflow for highly multiplexed reduced representation bisulfite sequencing](https://doi.org/10.1101/2020.06.23.137091) 7. [PopGLen-a Snakemake pipeline for performing population genomic analyses using genotype likelihood-based methods](https://pubmed.ncbi.nlm.nih.gov/40067089/) 8. [gyōza: a Snakemake workflow for modular analysis of deep-mutational scanning data](https://pubmed.ncbi.nlm.nih.gov/40976584/) 9. [ATLAS: a Snakemake workflow for assembly, annotation, and genomic binning of metagenome sequence data](https://pubmed.ncbi.nlm.nih.gov/32571209/) 10. [VIPER: Visualization Pipeline for RNA-seq, a Snakemake workflow for efficient and complete RNA-seq analysis](https://pubmed.ncbi.nlm.nih.gov/29649993/) 11. [HaploCharmer: a Snakemake workflow for read-scale haplotype calling adapted to polyploids](https://doi.org/10.1101/2025.03.14.642807) 12. [TriosCompass: a snakemake workflow for integrated detection of SNVs, indels, STRs, and structural de novo variants in parent-child trios](https://pubmed.ncbi.nlm.nih.gov/42162957/) 13. [Snakemake workflows for long-read bacterial genome assembly and evaluation](https://pubmed.ncbi.nlm.nih.gov/38591001/) 14. [A Snakemake Toolkit for the Batch Assembly, Annotation and Phylogenetic Analysis of Mitochondrial Genomes and Ribosomal Genes From Genome Skims of Museum Collections](https://pubmed.ncbi.nlm.nih.gov/39465511/)