FASTQ File Naming Conventions: A Standardized Approach

By Dr. Zubair Khalid, DVM, MS, PhD · 2026-11-11

By the end of this article you will be able to define a FASTQ naming convention for your own sequencing project, validate every file in a directory against that convention with a single regular expression, rename an existing batch of files without losing the link back to the original names, and confirm that your paired-end reads still pair correctly after the rename. You will need a Unix-like shell (Linux or macOS, or Windows Subsystem for Linux), Python 3.8 or later, GNU coreutils including md5sum, and a directory of FASTQ files to practice on. Nothing in this workflow modifies read data. It changes names, permissions, and directory structure only.

Why FASTQ File Naming Conventions Matter

A FASTQ filename is metadata. Before anyone opens a file, the name should answer four questions: which project produced the read data, which biological sample it came from, which sequencing run generated it, and whether it is the first or second read of a paired-end fragment. When names answer those questions consistently, downstream tools can parse them, collaborators can audit them, and future-you can reconstruct the dataset years later.

The evidence for consistency is not specific to genomics. A 2021 analysis in Nursing Economic$ found that standardized file naming improves retrieval efficiency and reduces the time staff spend locating files [1]. Researchers studying mobility data reached the same conclusion from the systems side: an explicitly defined naming convention prevented confusion and duplication of effort between database managers and analysts who had to exchange many files [2]. Anquetil and Lethbridge showed decades earlier that naming conventions alone are informative enough to cluster related files in legacy systems, which is essentially what a pipeline does when it groups _R1 and _R2 files into pairs. Recent work in the construction industry formalized the same idea for Building Information Modeling data exchange, arguing that a shared, ergonomically tested naming convention produces measurable efficiency gains across organizations [3].

In genomics the stakes are concrete. Workflow authors report that manual file naming and directory setup are among the most common sources of errors and failed resume attempts in RNA-seq pipelines, which is why tools such as FastqToGenCounts push naming into a configuration file rather than leaving it to the analyst at the keyboard [4]. A file named sample 3 (2)_R1 final.fastq.gz will break a Snakemake rule, a Nextflow channel, or a simple shell glob in ways that cost hours to debug. The fix is a convention adopted before data arrive, not after.

The FASTQ Format in Brief

FASTQ stores nucleotide sequences together with a per-base quality score. Each record occupies four lines: a header beginning with @, the sequence, a separator line beginning with +, and a quality string of the same length as the sequence. The format has no formal specification body, and it exists in at least three incompatible variants that differ in how quality scores are encoded. Cock and colleagues documented the Sanger (Phred+33) variant used by modern Illumina pipelines, the older Solexa and Illumina (Phred+64) variants, and the conversions between them, and this description is the de facto reference for the format [5].

Two practical consequences follow for naming. First, because multiple quality variants exist, a file that mixes data from different sequencing eras is a real hazard, so encoding the run or instrument generation in the filename is worthwhile. Second, because the four-line structure is the only thing every parser agrees on, the filename is the natural place to carry the sample identity that the format itself does not store.

The Default Illumina Filename, Explained

Most laboratories receive files that already follow a convention, because the bcl2fastq and DRAGEN demultiplexing software emits names in a fixed pattern:

SampleName_S1_L001_R1_001.fastq.gz

Reading the fields from left to right:

The read headers inside these files follow the CASAVA convention, with colon-separated fields for instrument, run number, flowcell identifier, lane, tile, x coordinate, y coordinate, read, filtered flag, and control number. This header structure is documented in Illumina technical materials and is standard knowledge for anyone processing bcl output.

The default names are unambiguous inside one run but incomplete across runs. Tumor5_S1_L001_R1_001.fastq.gz from a March run and the same name from a September run are different data with identical names. That collision, not the structure itself, is the problem a laboratory convention has to solve.

A Standardized FASTQ Naming Schema

Briney's File Naming Convention Worksheet describes a four-part process that transfers directly to sequencing data: choose the metadata that matters, encode and order that metadata, add version information, and format the final names for machine readability. Applied to FASTQ, that process yields the schema used for the rest of this article:

{ProjectID}_{SampleID}_{LibraryType}_{RunID}_{Read}_{Version}.fastq.gz

A filled-in example:

MYCO19_S023_RNA_R240115_R1_001.fastq.gz

Field-by-Field Specification

Character Set and Encoding Rules

Kodurupati's analysis of special characters in file naming shows that spaces, ampersands, parentheses, and similar symbols are a leading cause of failed pattern matching, because the same characters carry structural meaning in globs, regular expressions, and shell syntax [6]. The schema therefore restricts names to five safe characters: uppercase and lowercase letters, digits, the underscore, and the hyphen. Two rules follow.

Rule 1: no spaces, ever. A space forces every downstream script to quote variables, and one forgotten pair of quotes produces the classic broken-glob bug.

Rule 2: use the underscore only as a field separator and the hyphen only inside a field value. This separation is what allows a single regular expression to split a filename back into its fields, which is the property that makes automated pipelines possible.

Rule 3: encode dates as YYMMDD or YYYYMMDD, never as ambiguous month-day combinations. R240115 means January 15, 2024 and cannot be misread as the fifth of April.

Research guides from university libraries converge on the same character-set advice for general research data, recommending underscores over spaces and explicit, ordered elements in every filename [7] [8]. The schema above is that advice applied to the one piece of metadata genomics pipelines actually parse: the filename of a sequencing read file.

FASTQ File Naming Conventions Step by Step

This walkthrough takes a messy incoming directory and produces a validated, documented, checksummed dataset. Run each block from the project root.

Step 1: Inventory and Checksum the Raw Data

List every FASTQ file and count them:

find raw_data -type f -name "*.fastq.gz" | sort > file_inventory.txt
wc -l file_inventory.txt

Expected output for a 12-sample paired-end project:

24 file_inventory.txt

Twenty-four files for twelve samples means every sample contributed an R1 and an R2. A count of 23 means a mate is missing and you should resolve that before renaming anything.

Then record checksums so you can prove later that renaming changed no content:

md5sum raw_data/*.fastq.gz > md5_original.txt
head -n 2 md5_original.txt

Expected output:

d41d8cd98f00b204e9800998ecf8427e  raw_data/sample 3 (2)_R1.fastq.gz
9c4f2d1b77a1e0c8b3a5f6d2e1c0b9a8  raw_data/sample 3 (2)_R2.fastq.gz

Note that the original names contain spaces and parentheses. That is exactly the failure mode Kodurupati documents, where such characters break pattern matching downstream [6], and the checksum file preserves them long enough to verify content integrity after the rename.

Step 2: Write the Convention Down Before Renaming Anything

Create a file named naming_convention.md in the project root containing the field table from the previous section, two or three example filenames, and the controlled vocabulary for LibraryType. A convention that exists only in the analyst's head is not a convention. Briney's worksheet makes this step explicit for exactly that reason, treating the written encoding decision as part of the convention itself. Library data-management guides make the same point, requiring that the convention be documented where any collaborator can find it [9].

Include the validation pattern in the document as you write it. The pattern for this schema is:

^[A-Za-z0-9]{4,10}_[A-Za-z0-9-]{3,10}_(DNA|RNA|scRNA|mtDNA)_R[0-9]{6}_(R1|R2)_[0-9]{3}\.fastq\.gz$

Step 3: Build and Test a Validation Pattern

Count files that violate the convention:

grep -Evc '^[A-Za-z0-9]{4,10}_[A-Za-z0-9-]{3,10}_(DNA|RNA|scRNA|mtDNA)_R[0-9]{6}_(R1|R2)_[0-9]{3}\.fastq\.gz$' file_inventory.txt

Expected output while the data are still raw:

24

All 24 names fail, which is correct, because none of them follow the new schema yet. Now build the new names. Save the mapping script as rename_fastq.py:

#!/usr/bin/env python3
import re
import sys
from pathlib import Path

PATTERN = re.compile(
    r"^(?P<prefix>.+?)_?(R[2])(_00[1-9])?\.fastq\.gz$"
)
VOCAB = {"DNA", "RNA", "scRNA", "mtDNA"}
PROJECT = "MYCO19"
RUN = "R240115"
LIBTYPE = "RNA"

def build_name(path: Path) -> str:
    match = PATTERN.match(path.name)
    if not match:
        sys.exit(f"unparseable name: {path.name}")
    sample = re.sub(r"[^A-Za-z0-9-]", "", match.group("prefix"))
    if not sample:
        sys.exit(f"empty sample ID after cleaning: {path.name}")
    read = match.group(2)
    return f"{PROJECT}_{sample}_{LIBTYPE}_{RUN}_{read}_001.fastq.gz"

def main(directory: str, dry_run: bool) -> None:
    for path in sorted(Path(directory).glob("*.fastq.gz")):
        new_name = build_name(path)
        action = "WOULD RENAME" if dry_run else "RENAMING"
        print(f"{action}: {path.name} -> {new_name}")
        if not dry_run:
            target = path.with_name(new_name)
            if target.exists():
                sys.exit(f"target exists, refusing to overwrite: {target}")
            path.rename(target)

if __name__ == "__main__":
    main(sys.argv[8], dry_run="--apply" not in sys.argv)

The script has two deliberate safety properties. It refuses to overwrite an existing file, and the missing -apply flag makes the default behavior a dry run that prints actions without performing them.

Step 4: Rename with a Dry Run

python3 rename_fastq.py raw_data

Expected output:

WOULD RENAME: sample 3 (2)_R1.fastq.gz -> MYCO19_S3_RNA_R240115_R1_001.fastq.gz
WOULD RENAME: sample 3 (2)_R2.fastq.gz -> MYCO19_S3_RNA_R240115_R2_001.fastq.gz
WOULD RENAME: Tumor5_S1_L001_R1_001.fastq.gz -> MYCO19_Tumor5_RNA_R240115_R1_001.fastq.gz

Inspect every line. The dry run exists so that a mistake in the prefix extraction or the controlled vocabulary costs nothing. When the mapping is correct, apply it:

python3 rename_fastq.py raw_data --apply

Expected output:

RENAMING: sample 3 (2)_R1.fastq.gz -> MYCO19_S3_RNA_R240115_R1_001.fastq.gz

Step 5: Verify the Rename and Record the Mapping

Confirm that content survived the rename byte for byte. Re-run the checksum over the new names and compare checksum values, ignoring the filename column:

md5sum raw_data/*.fastq.gz | awk '{print $1}' | sort > md5_new_hashes.txt
awk '{print $1}' md5_original.txt | sort > md5_old_hashes.txt
diff md5_old_hashes.txt md5_new_hashes.txt && echo "CHECKSUMS MATCH"

Expected output:

CHECKSUMS MATCH

Then re-run the validation count. It should now report zero violations:

grep -Evc '^[A-Za-z0-9]{4,10}_[A-Za-z0-9-]{3,10}_(DNA|RNA|scRNA|mtDNA)_R[0-9]{6}_(R1|R2)_[0-9]{3}\.fastq\.gz$' file_inventory.txt

Expected output after regenerating the inventory:

0

Finally, save the old-name to new-name mapping. The dry-run output redirected to a file is that mapping, and it belongs in the project archive next to the sample sheet:

python3 rename_fastq.py raw_data > rename_map_2026-11-11.txt

This mapping file is what you consult when a collaborator asks for "the file that used to be called sample 3 (2)".

Step 6: Check that R1 and R2 Still Pair

A rename can silently separate mates if the read suffix is mishandled. Verify pairing by comparing read headers at the start of each file. The cut -c2- strips the leading @, and the first space-separated token is the read identifier:

diff <(zcat raw_data/MYCO19_S3_RNA_R240115_R1_001.fastq.gz | awk 'NR%4==1' | cut -d' ' -f1 | cut -c2- | head -n 1000) <(zcat raw_data/MYCO19_S3_RNA_R240115_R2_001.fastq.gz | awk 'NR%4==1' | cut -d' ' -f1 | cut -c2- | head -n 1000) && echo "PAIRS MATCH"

Expected output:

PAIRS MATCH

If the diff reports differences and your data came from an older pipeline that appends /1 and /2 to read names, add sed 's/\/$//' before the diff and compare again. If the diff reports differences that survive that adjustment, the mate files were mismatched before the rename, and the fix belongs with the sequencing provider, not in your script.

Step 7: Lay Out Storage and Lock It Down

Bioinformatics data management treats raw sequencing data as immutable. Adopt a three-tier layout:

project_root/
  00_raw/          read-only original FASTQ files and checksums
  01_metadata/     sample sheet, naming convention, rename map
  02_processed/    everything pipelines write

Make the raw tier resistant to accidental writes:

chmod -R a-w 00_raw

Expected result when someone later tries to edit inside it:

mv: cannot move '00_raw/MYCO19_S3_RNA_R240115_R1_001.fastq.gz' to '00_raw/new_name.fastq.gz': Permission denied

That permission error is the system working correctly. Reversing it requires the deliberate chmod -R u+w 00_raw, which is a decision, not an accident.

For genomics data storage at scale, the same layout principle applies whether the backing store is a laboratory server, a university cluster, or cloud object storage. Raw data are written once and never modified, processed data are regenerable from raw data plus code, and metadata explain how the two relate. Anquetil and Lethbridge's observation that names alone can cluster related files is what allows directory-level tools to operate on this tree without a database. Large archives in other domains have reached the same design, as in the naming convention built for a satellite imagery archive to keep terabyte-scale collections queryable by name alone.

Common Errors, Exact Messages, and Fixes

bash: No such file or directory after a glob. Cause: a filename contains a space or parenthesis, so the shell split it into multiple arguments. Fix: prevent the condition by enforcing the safe character set, and quote every variable in the interim: for f in *.fastq.gz; do md5sum "$f"; done.

gzip: stdin: unexpected end of file. Cause: the file was truncated during transfer, often by an interrupted copy. Fix: re-transfer and compare against the checksum manifest before renaming. Never rename a file that fails its checksum, because the new name would lend false confidence to corrupt data.

find: paths must precede expression. Cause: an unquoted or misordered argument to find, frequently a pattern with special characters expanding unexpectedly. Fix: quote the pattern (find . -type f -name "*.fastq.gz") and run find after the character-set rules are in force.

Aligner aborts with a mate-count mismatch. Cause: R1 and R2 files contain different numbers of reads, usually because one mate was renamed into the wrong sample slot. Fix: re-run the pairing check from Step 6 on the two suspect files, and consult the rename map rather than guessing.

target exists, refusing to overwrite: ... from the rename script. Cause: two source files collapse onto the same target name, which means the sample IDs collided after cleaning. Fix: inspect both source names, extend the SampleID field definition, and re-run the dry run. This check is why the script exits rather than calling rename unconditionally.

Validation pattern reports a violation for a legitimate file. Cause: the regex vocabulary (DNA|RNA|scRNA|mtDNA) does not include a LibraryType that appeared late in the project. Fix: extend the controlled vocabulary in both the convention document and the pattern, then re-validate. A convention that cannot absorb a new assay type gets bypassed, and a bypassed convention is worse than none.

Common Mistakes and Limitations

Renaming without a dry run. mv overwrites silently on most systems. A single transposed character in a sed expression can replace 24 files with 1. The dry run costs seconds and prevents that class of accident entirely.

Renaming without checksums. If you cannot prove that content survived the rename, you cannot troubleshoot a downstream result that looks wrong. The checksum comparison in Step 5 is the audit trail.

Changing the sample identifier meaning halfway through a project. If S023 means "sample 23" in March and "pool 23, plate 2" in August, every merged analysis is compromised. Fix the meaning in the convention document and never amend it retroactively.

Packing too much metadata into the filename. Filenames should carry identity, not the full experimental design. Treatment, batch, and protocol details belong in a sample sheet. Cross-domain work on naming conventions, including the construction-industry study, shows that overloading names with information degrades both human readability and parsing reliability [3].

Assuming case insensitivity. Linux filesystems distinguish MYCO19 from myco19 while the default macOS filesystem does not. A project that looks consistent on a Mac can produce duplicate-looking samples on a cluster. Pick one casing rule and validate for it.

Trusting _001 as a version number. In bcl2fastq output, 001 is the split segment number, not a processing version. The schema above reuses three-digit zero padding for versions because the shape is familiar, but it documents a different meaning. Confusing the two is a recurring source of bad merges.

The convention is a project-level agreement, not a universal standard. No governing body mandates a single FASTQ filename grammar, and instrument vendors, repositories such as the Gene Expression Omnibus, and pipeline frameworks each impose their own partial requirements. The value of the schema here is that it is explicit, machine-parseable, and documented, which is what the naming-convention literature across domains consistently identifies as the effective ingredient.

Designing the Convention: Choosing Fields for Your Project Type

The six-field schema in this article is a starting point, not a mandate. The fields you encode should follow from the questions your own group asks six months after sequencing, and Briney's four-part worksheet makes that explicit: choose the metadata that matters, encode and order it, add versioning, then format for machines. The most common design error is the opposite of the one people expect. Laboratories rarely omit too little metadata; they encode too much, and the filename becomes an unreadable string that humans copy with errors and scripts parse with exceptions.

A useful filter for candidate fields: a field earns a place in the filename only if (a) pipelines or people need it without opening a metadata file, and (b) its value is stable for the life of the file. Apply that filter to three common project shapes and the differences become obvious.

A clinical sequencing panel. The questions asked later are almost always "which patient specimen" and "which assay run." Sample identity is usually a pseudonymized accession, so the schema might be:

PANEL_PT-0451_PANEL_R240402_R1_001.fastq.gz

Here the project code and library type are nearly redundant, because the laboratory runs one assay and one project. Resist the urge to delete them anyway. Redundant fields cost four characters and buy you the ability to merge this directory with another project later without renaming anything.

A longitudinal cohort study. The recurring question is "which visit," so a timepoint field earns its place:

GLUC_S112_T3_RNA_R241108_R1_001.fastq.gz

Note what did not earn a place: treatment arm, age, batch, fasting status. Those change the analysis but never the file's identity, and they belong in the sample sheet keyed by GLUC_S112_T3. University library guides on research data naming make the same division: the name carries identity and ordering, the metadata record carries description [7] [8].

A core facility serving many laboratories. Here a PI or billing code field is often mandatory, because the recurring question is "whose data is this" when three deliveries land the same week:

ZKLAB_MYCO19_S023_RNA_R240115_R1_001.fastq.gz

Whatever variant you choose, write it down before the first delivery arrives. A one-page convention document needs exactly five things: the field order table, the controlled vocabulary for every closed-set field, three example filenames including one that should fail validation, the validation regex, and the date format. Library data-management guides require documentation as part of the convention itself for a simple reason: a convention stored in one analyst's head cannot be audited, taught, or defended during a data review [9]. Pair the document with a short decision log recording why each field was included and why others were rejected. That log is what prevents a well-meaning colleague from "simplifying" the schema mid-project and breaking every pipeline that parses it.

One more design rule worth stating plainly: field order is fixed forever once data exist. If you reorder fields after a hundred files are named, every regex, every cut -d'_' command, and every mapping file becomes ambiguous. If a field must be added, append it before the extension or between existing fields with a documented migration, and re-validate the entire directory afterward.

Worked Example: A Mixed Single-Cell Delivery

Single-cell deliveries stress a naming convention harder than bulk RNA-seq because one biological sample often produces several library types in one run, and the vendor default names them all S1, S2, and so on with no visible sample identity. Suppose a core delivers a ten-sample 3' expression run plus the corresponding feature barcode libraries:

GEX-0123_S1_L001_R1_001.fastq.gz
GEX-0123_S1_L001_R2_001.fastq.gz
FB-0123_S1_L001_R1_001.fastq.gz
FB-0123_S1_L001_R2_001.fastq.gz

The problem is that S1 in the vendor name means "first row of the sample sheet," and the biological sample ID lives only in the prefix, which here is a library barcode label rather than a sample. The fix is a per-delivery mapping decision, recorded before renaming: which vendor sample numbers correspond to which study sample IDs. That mapping comes from the core's sample sheet, and if the core did not provide one, obtain it before renaming. Guessing the mapping from filenames alone is the single most dangerous step in this whole workflow.

Adapt the rename script with two changes. The LIBTYPE constant becomes a lookup keyed on the parsed prefix, and the controlled vocabulary expands:

LIBMAP = {
    "GEX": "scRNA",
    "FB":  "FB",
}

The dry run now prints mappings for both library types per sample:

WOULD RENAME: GEX-0123_S1_L001_R1_001.fastq.gz -> MYCO19_S023_scRNA_R240115_R1_001.fastq.gz
WOULD RENAME: FB-0123_S1_L001_R1_001.fastq.gz  -> MYCO19_S023_FB_R240115_R1_001.fastq.gz

After applying the rename, run three verification passes in order. First, checksums, exactly as in the main workflow. Second, the regex validation count against the updated vocabulary. Third, a per-sample read-count equality check, which matters more for single-cell data because the expression and feature barcode libraries must describe the same cells:

for r1 in raw_data/*_R1_001.fastq.gz; do
  r2="${r1/_R1_/_R2_}"
  n1=$(zcat "$r1" | awk 'END{print NR/4}')
  n2=$(zcat "$r2" | awk 'END{print NR/4}')
  if [ "$n1" != "$n2" ]; then
    echo "COUNT MISMATCH: $r1 ($n1) vs $r2 ($n2)"
  fi
done

Expected output for a clean delivery is silence. Two honest caveats about this loop. It assumes every record ends with a newline, so a file truncated mid-record can still report a matching count; that is why the header-level diff from Step 6 of the main workflow remains the authoritative pairing test, and this loop is the fast screening version. It also assumes the Read field is always exactly _R1_ or _R2_ surrounded by underscores, which the shell substitution exploits safely only because the RunID field, though it starts with R, never equals the literal string R1.

Worked Example: A Longitudinal Project with Resequence Runs

Sooner or later a sample fails quality thresholds and the core resequences it. Now the same SampleID legitimately has two sets of files, and the RunID field is what keeps them apart. Consider S023 sequenced in January and again in February:

MYCO19_S023_RNA_R240115_R1_001.fastq.gz
MYCO19_S023_RNA_R240220_R1_001.fastq.gz

Both names validate against the same regex. Both are correct. The design question is what happens downstream, and the convention should answer it in writing before the resequence arrives, because there are two defensible strategies and they must not be mixed within a project.

Strategy one: treat every run as a separate analysis unit. Each run's files proceed independently through QC and quantification, and the sample sheet records that S023 has two analytical replicates. This is the safer default because it never destroys information and it exposes run effects rather than hiding them.

Strategy two: merge the raw reads into one file set per sample. Some analyses genuinely want all reads in one place. If your project chooses this, the merge is a new processing event, and the Version field is where it is recorded:

zcat 00_raw/MYCO19_S023_RNA_R240115_R1_001.fastq.gz \
     00_raw/MYCO19_S023_RNA_R240220_R1_001.fastq.gz \
  | gzip -c > 02_processed/MYCO19_S023_RNA_MERGED_R1_002.fastq.gz

Note three things about that command. It writes into the processed tier, not the raw tier, because the merged file is derived data and the raw tier is read-only. It uses the _MERGED_ position where RunID would sit, which is legal only because the convention document now defines that variant explicitly; an undocumented variant is exactly the kind of drift that makes validation regexes lie. And the checksum of the merged file is recorded alongside the checksums of its two inputs, so the provenance chain reads: two checksummed raws, one derived file, one line in the decision log explaining the merge.

The same logic handles partial resequences where only a top-up lane was run. The tempting move is to rename the top-up files as if they were the original run, which makes the two deliveries look like one delivery and quietly corrupts any per-run quality metric. The convention's job is to make that mistake impossible: two runs get two RunIDs, always, and any pooling happens as a documented, versioned, derived event.

Extending the Schema for Lanes, Split Files, and Single-End Data

The six-field schema assumes one file per read per sample, which is true when demultiplexing runs with lane splitting disabled. It is frequently not true. With lane splitting enabled, one sample yields four lane files per read, and large-flowcell outputs are sometimes further split into segments. Two extension patterns cover both, and choosing between them is a one-time project decision.

Pattern A: keep the vendor names in the raw tier and rename only working copies. The raw tier stores Tumor5_S1_L001_R1_001.fastq.gz exactly as delivered, and the processed tier holds standardized copies with a lane field inserted between RunID and Read:

MYCO19_Tumor5_RNA_R240115_L001_R1_001.fastq.gz

The extended validation pattern adds one field:

^[A-Za-z0-9]{4,10}_[A-Za-z0-9-]{3,10}_(DNA|RNA|scRNA|mtDNA)_R[0-9]{6}_L[0-9]{3}_(R1|R2)_[0-9]{3}\.fastq\.gz$

Pattern A costs double storage for the raw copies but has a strong advantage: the raw tier is always a byte-identical mirror of the core's delivery, so any dispute about what was received is settled by checksum against the delivery manifest.

Pattern B: rename in place and record the lane mapping in the rename map file. This halves storage but means the vendor lane structure survives only in the mapping document. For most projects with modest storage, Pattern B is fine; for clinical or regulated work, Pattern A's redundant raw mirror is usually worth the disk.

Single-end data need one explicit decision, not a silent one. Either the Read field is always present and always R1, or it is omitted for single-end libraries. Pick the first option. A pipeline that must handle both _R1 and no-read-suffix names has two grammar branches instead of one, and every downstream parser pays for that branch forever. Uniformly emitting R1 for single-end data keeps the regex single-shaped, which is the property that makes one-line validation possible at all.

Finally, resist adding fields you do not parse. The satellite imagery archive cited earlier keeps terabyte-scale collections queryable by name because its fields were chosen for the queries actually run against them. The same discipline applies here: every field you add must appear in at least one command, one pipeline rule, or one recurring question. A field nobody parses is a field everybody has to skip, and skipping logic is where parsing bugs are born.

A Regex Deep Dive: Building the Validation Pattern Field by Field

The one-line validation command is the most valuable artifact in this workflow, and it deserves to be understood token by token rather than copied blindly. Take the pattern apart:

^
[A-Za-z0-9]{4,10}      ProjectID: 4 to 10 alphanumerics
_
[A-Za-z0-9-]{3,10}     SampleID: 3 to 10, hyphens allowed inside
_
(DNA|RNA|scRNA|mtDNA)  LibraryType: closed set, alternation
_
R[0-9]{6}              RunID: R then exactly six digits
_
(R1|R2)                Read: closed set
_
[0-9]{3}               Version: three digits
\.fastq\.gz
$

Four details do most of the work, and each corresponds to a failure mode worth knowing.

The anchors. ^ and $ are why the pattern validates whole names. Without them, grep -E reports a match if the pattern appears anywhere in the line, and a file named MYCO19_S023_RNA_R240115_R1_001.fastq.gz.backup would sail through validation while being unusable by every pipeline. When debugging, temporarily remove the anchors and use grep -oE to see which substring matched; the difference between the matched substring and the full name points directly at the violating field.

The escaped dots. \. matters because an unescaped . matches any character, so fastq.gz without escaping would also accept fastqXgz and, more insidiously, fastq gz if the filename contained a space. This is the same class of problem Kodurupati documents, where characters with structural meaning in pattern languages cause silent misbehavior rather than loud failures [6].

The bounded quantifiers. {4,10} and {3,10} are not decoration. They encode the convention document's field-length rules, and they catch the specific accident where a script concatenates two fields and produces a name that looks plausible at a glance. A pattern with only [A-Za-z0-9]+ would accept a 60-character ProjectID without complaint; the bounded version rejects it. Note also the choice of [7] rather than a digit shorthand: inside grep -E, POSIX classes and backslash shorthands behave differently across implementations, and [7] is the spelling that behaves identically everywhere.

The alternation placement. (DNA|RNA|scRNA|mtDNA) uses grouping parentheses so the alternation is confined to one field. A common drafting error is writing _DNA|RNA_ without parentheses, which the regex engine reads as two complete alternatives, and then the pattern accepts nonsense like MYCO19_S023_DNA_R240115_RNA_001.fastq.gz because it matches the second alternative's tail. If your validation count suddenly reports fewer violations than the dry run produced renames for, an ungrouped alternation is a likely culprit.

To test the pattern while drafting, feed it single examples rather than whole directories:

echo "MYCO19_S023_RNA_R240115_R1_001.fastq.gz" | grep -Ec 'pattern'

A count of 1 means the name validates. Build the pattern against three canonical names from the convention document, one valid, one with a bad LibraryType, one with a lowercase ProjectID, and only then point it at the real directory. This draft-against-examples habit costs two minutes and catches most pattern bugs before they touch data.

Parsing Convention-Compliant Names in Pipelines

A naming convention pays for itself the first time a pipeline extracts fields from filenames instead of asking a human to type sample lists. Once every file matches the schema, field extraction is deterministic, and there are three standard techniques worth having ready.

Shell, using the underscore as a split character. This is where Rule 2, underscores only between fields and hyphens only within values, earns its keep:

for f in raw_data/*.fastq.gz; do
  b=$(basename "$f")
  project=$(echo "$b" | cut -d'_' -f1)
  sample=$(echo "$b" | cut -d'_' -f2)
  libtype=$(echo "$b" | cut -d'_' -f3)
  echo "$sample $libtype $f"
done

The same extraction in pure parameter expansion, which avoids one subprocess per field:

b="MYCO19_S023_RNA_R240115_R1_001.fastq.gz"
project=${b%%_*}
tail=${b#*_}
sample=${tail%%_*}

Building the sample list. Almost every workflow begins by deriving the set of samples from the directory:

ls raw_data/*_R1_001.fastq.gz | xargs -n1 basename | cut -d'_' -f2 | sort -u > samples.txt

Read that command as a contract: because the Read field is fixed, selecting _R1_ files yields exactly one file per sample per run, and sort -u collapses repeats across runs. If your convention allowed the read field to be optional, this command would double-count single-end samples, which is a concrete example of why uniformity beats flexibility in filenames.

Deriving the mate filename. Pipelines constantly need the R2 path given the R1 path. The safe substitution replaces the full field, underscores included:

r2="${r1/_R1_/_R2_}"

The underscores on both sides are not cosmetic. A replacement of bare R1 with bare R2 would also corrupt a sample ID such as PR1-7, silently pointing the aligner at the wrong mate file. Field-delimited substitution is immune to that collision, and it is one more dividend of the discipline that field separators and field contents use disjoint characters.

Workflow frameworks follow the same logic with different syntax. The practical guidance is identical in Snakemake, Nextflow, or a plain Makefile: constrain any sample wildcard to match a single field rather than the whole name, derive sample lists from R1 files only, and construct mate paths by field substitution rather than by string surgery on the whole filename. This is the same principle the RNA-seq tooling literature identifies when it pushes file naming into configuration rather than improvisation at the keyboard [4]. A pipeline that treats the filename grammar as an interface, and validates that interface once at intake, inherits all of the convention's guarantees; a pipeline that free-parses arbitrary names inherits none of them.

Comparing Naming Approaches: Ad Hoc, Vendor Default, and Standardized

It helps to see the three realistic options side by side, because each is genuinely the right choice somewhere, and the mistake is picking by habit rather than by fit.

Ad hoc names, meaning whatever the bench scientist or the instrument's export dialog produced, are acceptable only for a single analysis by a single person with a short memory, and even then the 2021 evidence on retrieval time suggests the cost of inconsistency compounds quickly as file counts grow. The vendor default is a good intermediate: it is parseable, pair-aware, and consistent within a run, which makes it dramatically better than ad hoc naming with zero effort. Its two failures, cross-run collisions and missing project context, are precisely the two failures that matter for any dataset that outlives one analysis or one disk.

The standardized schema's costs deserve honest statement. Adoption requires a renaming pass, a mapping file, and a maintained document, and those are recurring costs whenever a new person or a new delivery joins the project. The standardized approach is worth that cost when any of the following is true: the project spans multiple sequencing runs, more than one person will touch the data, the data may be shared or published, or pipelines parse filenames. If none of those is true, the vendor default with a checksum manifest is a defensible choice, and pretending otherwise oversells the convention. What the cross-domain literature consistently shows is not that one grammar is universally superior, but that explicitness and consistency produce measurable gains wherever files are exchanged between people or between software [3]. The six-field schema is one explicit, consistent answer, sized for genomics.

Common Misconceptions About FASTQ Naming

Several persistent beliefs cause real damage in practice, and each is worth correcting precisely.

"The filename is the metadata." The filename is one piece of metadata, and a thin one. Treatment, batch, extraction date, library protocol, and read length all live in the sample sheet. A group that loads its sample sheet by parsing filenames has coupled two systems that should be loosely coupled, and the coupling breaks the first time a filename must be corrected. The convention exists so the filename can act as a foreign key into richer metadata, not so it can replace the metadata.

"The .gz extension proves the file is valid gzip." The extension is a naming promise, not a content guarantee. A truncated transfer produces a .fastq.gz file that many tools accept until they hit the truncated tail. gzip -t file.fastq.gz verifies the archive structure, and the checksum manifest verifies content; neither substitutes for the other.

"Renaming risks corrupting the data." Renaming changes a directory entry, not file contents, and the checksum comparison proves it. The real rename risks are operational: overwriting an existing target, dropping a mate, or renaming before the checksums are recorded. Those are process failures with process fixes, namely the dry run, the refuse-to-overwrite check, and the checksum-before-rename ordering.

"Case does not matter." It matters differently on different filesystems, and the failure is asymmetric. A project developed on a case-insensitive default macOS filesystem can harbor two files differing only in case, which coexist peacefully on the Mac and then collide, or fail to both arrive, when transferred to a Linux cluster. Pick a casing rule, encode it in the regex, and validate on the filesystem where the analysis will actually run.

"Underscores and hyphens are interchangeable." They are not, once the convention assigns them structural roles. Swapping them in one file turns that file into a regex violation and, worse, into a name that splits into the wrong number of fields under cut -d'_'. The separation of separator and content characters is the single design decision that makes one-line validation possible, so it is the one rule with no exceptions.

"The format's variants make filenames irrelevant." The three quality-encoding variants of FASTQ documented by Cock and colleagues are a content-level hazard, and the naming convention addresses them only indirectly, by carrying the RunID that lets you trace which instrument generation produced a file [5]. That indirect handle is genuinely useful, but it is not a substitute for recording encoding details in the metadata when data from different eras share a project.

"Once the convention exists, it maintains itself." Conventions decay by drift: one late delivery renamed by hand, one new assay type added to the vocabulary in a script but not the document, one collaborator who "just quickly" fixed a name. The maintenance section below treats the convention as a living artifact with scheduled checks, because that is what keeps it from becoming another ad hoc scheme with extra steps.

Troubleshooting Collisions, Late Arrivals, and Returned Files

The rename workflow in the main article covers a clean delivery. Real projects produce messier situations, and each has a standard resolution.

Two source files map to the same target name. The script's refuse-to-overwrite check fires, which means cleaning removed the only distinguishing characters between two sample IDs, for example sample 3 and sample-3 both becoming S3. Do not "fix" this by renaming one file by hand. Resolve the identity question against the sample sheet: either the two files genuinely belong to one sample, in which case they are split segments and should be concatenated or versioned, or they are distinct samples whose IDs need a documented disambiguation, in which case extend the SampleID definitions, update the convention document, and re-run the dry run for the whole directory.

A late delivery arrives mid-project. The temptation is to fold it in with a quick manual rename. Resist it, because manual renames skip the checksum and mapping steps that make the rest of the archive auditable. The late delivery gets the full workflow: inventory, checksums, dry run, apply, verification, and an entry appended to the rename map with its own date. Because the schema includes the delivery's own RunID, late arrivals never collide with earlier files even when the sample IDs repeat.

A collaborator returns files with new names. This is commoner than anyone admits, and it is the scenario where the rename map pays for itself. The returned files cannot be validated against your schema, but their content can be identified: compute checksums of the returned files and match them against the checksum manifest. A checksum match proves the returned file is byte-identical to one you shipped, and the manifest plus rename map tells you exactly which sample and run it belongs to. No name matching, no guessing, no ambiguity.

Files arrive with the read suffix appended in the headers, such as /1 and /2. This is a header convention difference, not a naming failure, and the pairing check in the main workflow already includes the adjustment for it. The one trap is assuming all files in a mixed delivery share the header style. Run the pairing check per file pair, not once for the whole directory, and record the header style of each delivery in the convention document's decision log.

Validation passes but a pipeline still fails on a name. Check for invisible characters. A filename that pasted through a word processor or a messaging application can carry a non-breaking space or a Unicode lookalike that renders identically and fails every ASCII-range regex. ls | od -c | head on the offending listing, or Python's repr() on the name, exposes the hidden bytes. The fix is to retype the name locally with the keyboard, then re-validate, and the prevention is to never let filenames travel through rich-text channels.

The checksum manifest itself goes missing. Without it, the rename audit trail weakens but does not vanish: the rename map still records original names, and if the raw tier was made read-only before any renaming, the modification timestamps plus the map reconstruct most of the provenance. This is a reason to store md5_original.txt, the rename map, and the convention document together in the metadata tier, so that losing one means losing the directory, not losing one file out of three.

Keeping the Convention Alive: Audits, Onboarding, and Archive

A convention is a piece of laboratory infrastructure, and infrastructure that is never inspected degrades silently. Three lightweight practices keep it intact.

A scheduled audit script. Once per project phase, or quarterly for long studies, run a single audit that re-establishes all guarantees. It has four checks, each of which already exists as a command in this article: regenerate the file inventory, count regex violations, compare a spot sample of checksums against the manifest, and run the read-count pairing loop. Wrap them in one script, print a one-line verdict per check, and save the output with a date in the filename. A ten-minute audit that produces a dated, fileable record converts "I think the raw data are fine" into evidence, and it catches drift within one quarter instead of at publication time.

Onboarding by the document, not by the person. When a new student or analyst joins, their first task on the project is to read the convention document and then run the audit script. This serves double duty: the newcomer learns the directory structure and the validation tooling in one session, and their fresh eyes are the cheapest available review of whether the document still matches reality. If the newcomer finds a file the regex rejects, the document and the directory disagreed, and the discrepancy is now cheap to fix. Library data-management guidance emphasizes exactly this placement of the convention where any collaborator can find and apply it [9], and the onboarding ritual is the enforcement mechanism that keeps it there.

Archival as a named unit. When a project reaches its end, the archive is a single directory containing the raw tier with its checksums, the metadata tier with the sample sheet, convention document, decision log, and every rename map, and a manifest listing all of the above. The cluster of related files that Anquetil and Lethbridge showed can be reconstructed from names alone is precisely what makes such an archive self-describing years later, without a database or the memory of the person who built it. Before sealing the archive, run the audit script one final time and store its output inside. An archive that includes its own passing audit is verifiable by whoever opens it, which is the entire point of standardization in the first place.

Sample Sheet Sync: Checking Names Against Metadata

The final integration point is the sample sheet, and the check that closes the loop is a set comparison: every sample ID appearing in filenames should appear in the sheet, and every sheet row should have at least one file. The comparison is two commands with comm:

ls raw_data/*.fastq.gz | xargs -n1 basename | cut -d'_' -f2 | sort -u > names_in_files.txt
cut -f2 01_metadata/sample_sheet.tsv | tail -n +2 | sort -u > names_in_sheet.txt
comm -3 names_in_files.txt names_in_sheet.txt

Expected output for a fully synced project is silence. Read the three possible outputs as diagnostics. A name in the left column only exists in filenames but not the sheet, which usually means a delivery arrived without a sheet update, and the fix is a sheet edit, not a rename. A name in the right column only exists in the sheet but has no files, which usually means a failed or partial delivery, and the fix is a call to the sequencing core before anyone writes a pipeline rule that assumes completeness. An output line appearing in both columns is impossible under comm -3, so any garbled line usually means the field positions were wrong: the sample ID is not column 2 of the sheet or not field 2 of the name, and the extraction commands, not the data, are at fault.

Two honest caveats. This comparison assumes the sheet's sample IDs match the filename's SampleID field character for character, including case and hyphenation, so normalize both sides through the same cleaning rule if the sheet predates the convention. And the comparison says nothing about whether the right reads are in the right files, only that the identifier sets agree; the pairing and checksum checks from the main workflow cover the content level. Together, the three checks, set equality, read-count equality, and checksum equality, form a layered verification where each layer catches a class of error the others cannot, which is the practical summary of everything a naming convention is for: not preventing every mistake, but making every remaining mistake visible and traceable to its origin.

Frequently Asked Questions

What is the standard naming for Illumina FASTQ files?

The bcl2fastq and DRAGEN convention is SampleName_S#_L00#_R#_00#.fastq.gz, where the fields are the sample name from the sample sheet, the sample number, the lane, the read number, and the split segment. It is unambiguous within a single run but collides across runs, which is why laboratory conventions extend it with project and run identifiers.

Should I rename the files my sequencing core delivered?

Only if you also record the original names. Keep a checksummed rename map so every analysis can be traced back to the deliverable, and keep an untouched copy of the raw delivery if storage permits.

What does the _001 suffix in a FASTQ filename mean?

In Illumina demultiplexer output it is the segment number, incremented when one sample's output is split across multiple files. Many laboratories reuse the three-digit shape as a processing version in their own conventions, which is fine as long as the convention document says so explicitly.

Are R1 and R2 files interchangeable?

No. R1 holds read 1 of each fragment and R2 holds read 2, and aligners require the correct assignment. Swapping them produces silent or cryptic failures, which is why the pairing check in this workflow compares read identifiers between the two files.

Do FASTQ files need the .gz extension?

The gzip extension is not part of the FASTQ format itself, but practically all pipelines expect compressed files and most tools read .fastq.gz natively. Name compressed files with the full .fastq.gz suffix rather than .fq alone.

How long can a FASTQ filename be?

Most modern filesystems allow 255 bytes for the filename component, which is far more than any useful FASTQ name needs. Keep names under roughly 80 characters so they remain readable in terminal listings, log files, and reports without wrapping.

Which characters are safe to use in a FASTQ filename?

Letters, digits, the underscore, and the hyphen are safe across Linux, macOS, and Windows, and they carry no special meaning in shell globs or regular expressions. Spaces, parentheses, ampersands, and similar symbols break pattern matching and should be avoided [6].

How do I check that R1 and R2 files pair correctly?

Decompress the first few thousand records of each file, extract the read identifier from every fourth line, and compare the two identifier lists with diff. Identical lists confirm pairing, and mismatches identify a swapped or mismatched mate before it reaches an aligner.

Sources

1. Improve Efficiency with Use of Standardized File Naming Conventions
2. Towards an Affordable GIS for Analysing Public Transport Mobility Data: A Preliminary File Naming Convention for Avoiding Duplication of Efforts
3. Efficient File-Naming Convention for Data Exchange in Construction Industry
4. FastqToGenCounts: Converting Fastq Files to Gene Counts Matrices
5. The Sanger FASTQ file format for sequences with quality scores, and the Solexa/Illumina FASTQ variants
6. Handling Special Characters in File Naming Conventions for An Error-Free Pattern Matching Approach
7. Research Guides: LabArchives: Best practices for research data management: File-naming conventions
8. Library Guides: UPRISE Data Management 101 Guide: File Naming Conventions
9. LibGuides: Engineering and Public Policy: Data Management: File Naming Conventions