FASTQ Header Format Error: Causes and Fixes

By Dr. Zubair Khalid, DVM, MS, PhD ·

FASTQ Header Format Error: Causes and Fixes

The most common cause of a FASTQ header format error is a file whose first line of each record does not begin with an at sign followed by a usable identifier. In practice that means one of three things: the file is not FASTQ at all, it was truncated during transfer, or a tool rewrote the identifiers into a dialect your current parser will not accept. The first thing to check takes about ten seconds. Run head -4 reads.fastq and read the four lines with your own eyes. Line 1 should start with @. Line 3 should start with +. Line 4 should be exactly as long as line 2. If any of those three conditions fails, you have already located the class of problem, and the rest of this guide is about confirming the specific cause and repairing it without damaging the data.

FASTQ header errors are frustrating because the message you see is usually generated by the parser, not by the file. A tool reports "invalid FASTQ header" or "malformed record" and stops, but the actual defect may be a carriage return introduced by a Windows text editor, a gzip stream that ended early, or a quality line that was shifted by one character during a merge. Treat the error message as a pointer to a record, not as a diagnosis. The diagnosis comes from inspecting the raw bytes.

This guide covers the format itself, the failure modes in order from most common to rarest, a diagnostic sequence you can run without special software, and a set of repairs that keep the original file intact. It also covers the boundary between FASTQ and FASTA, because a large share of header errors are simply the wrong file type arriving at the wrong step.

What a FASTQ Record Actually Contains

FASTQ is a plain-text format for storing a nucleotide sequence together with one quality score per base. That pairing is the entire point of the format, and it is why FASTQ replaced FASTA for raw sequencing reads in the mid-2000s. The format was popularized by the Wellcome Trust Sanger Institute and later formalized in a widely cited description of the Sanger, Solexa, and Illumina variants.

The four-line structure

A FASTQ record occupies exactly four lines when written in its canonical form.

  • Line 1 begins with @ and carries the read identifier and optional description.
  • Line 2 is the sequence, written 5-prime to 3-prime.
  • Line 3 begins with +. It may be bare or may repeat the line 1 identifier. Most tools ignore whatever follows the plus.
  • Line 4 is the quality string, one ASCII character per base in line 2.

A minimal record looks like this.

@SEQ_ID_1
GATTACAGATTACA
+
IIIIIIIIIIIIII

The quality string is not stored as numbers. Each character encodes a Phred score after subtracting an offset. In the modern standard, the offset is 33, so ! is Q0, + is Q10, I is Q40, and ~ is Q93. The printable ASCII range from ! (33) to ~ (126) covers the whole practical score range. Older Illumina pipelines used an offset of 64, which is why encoding mismatches still appear in archived data.

Because the format is line-oriented and position-based, a parser does not search for the next @. It reads four lines, treats the first as a header, and moves on. That design decision is efficient and it is also the reason single-byte corruption produces cascading errors.

What lives in the header

The FASTQ specification does not mandate a header schema beyond the leading @. Header content is a de facto convention, and the conventions differ by sequencing platform and by the pipeline that demultiplexed the run. Three dialects account for most public data.

The Illumina Casava 1.8 convention, used by bcl2fastq2 and later, packs instrument, run, flowcell, lane, tile, coordinates, read number, filter flag, control bits, and index sequence into a colon-delimited identifier, followed by a space and a read block.

@M01234:56:000000000-ABCDE:1:1101:14022:1876 1:N:0:ACGTACGT

The older Casava 1.7 convention uses a hash and a slash.

@HWUSI-EAS100R:6:73:941:1973#0/1

The SRA and ENA conventions use an accession with a dot and a version number, optionally followed by a free-text description.

@SRR001666.1 071112_SLXA-EAS1_s_7:5:1:817:345 length=36

A parser that expects one dialect and receives another may accept the record but mis-extract the read number, or it may reject the header outright. Many downstream tools need to know whether a record is read 1 or read 2, and they infer that from the slash suffix, the space-delimited read block, or a convention imposed by the file name.

Where the header error usually appears

Header errors surface in four places. Quality control tools report them during parsing. Trimmers and adapters report them when they cannot map a record to a pair. Aligners report them when they need a read group. Workflow managers report them when a task's output fails validation and the pipeline aborts. In every case the failure is at the boundary between a producer and a consumer, and the fix belongs at the boundary, not deep inside the pipeline.

FASTQ versus FASTA and the Wider Family of Bioinformatics File Formats

FASTA is the older and simpler member of the family. A record begins with > instead of @, the sequence may be wrapped across any number of lines, and there are no quality scores at all. FASTA is the right format for reference genomes, protein sequences, and any data where per-base confidence is not part of the record.

The practical consequences for header errors are direct. Any tool that expects FASTQ will reject a FASTA file on the first byte, because > is not @. Any tool that expects a four-line record will misparse a wrapped FASTA file even if you rename the extension. Changing a file extension does not change a file format.

The two formats also differ in what they can lose. Converting FASTQ to FASTA is lossless for the sequence and lossy for the quality. Converting FASTA to FASTQ requires inventing quality scores. The common trick is to write I for every base, which claims Q40 for every position. That claim is false, and it propagates into every downstream filter, trim, and variant model that reads the quality string. If a tool demands FASTQ and you only have FASTA, check whether the tool has a documented FASTA mode before you fabricate qualities.

Bioinformatics file formats form a rough hierarchy. FASTA and FASTQ hold reads and references as text. SAM holds alignments as text with a mandatory header section. BAM and CRAM hold the same alignments in compressed binary form. VCF and BCF hold variants. FASTQ sits at the front of that chain as the container for raw instrument output, which is why its header conventions are the least standardized and the most fragile.

Troubleshooting Table: Symptom to Fix

SymptomLikely causeHow to confirmFix
Line 1 starts with > instead of @File is FASTA, not FASTQhead -1 reads.fastqUse the correct input or convert deliberately, see below
Line 1 starts with @SQ or @HDFile is SAM, not FASTQhead -5 shows a SAM header blockExtract reads with samtools fastq
Parser stops partway through, no obvious bad lineTruncated download or interrupted writegzip -t reads.fastq.gz and compare line count modulo 4Re-download and verify the checksum
file reports CRLF line terminatorsWindows line endingsfile reads.fastqStrip carriage returns with tr -d '\r'
Sequence and quality lengths differWrapped record or shifted quality lineawk length comparison at every fourth lineUnwrap with a normalizing parser or discard the record
Tool rejects a valid-looking headerHeader dialect mismatchInspect the space-delimited fields after the identifierNormalize headers at ingest
Duplicate identifier error on mergeTwo libraries share read namesawk plus sort plus uniq -dPrefix identifiers with a sample tag
Invalid quality characterEncoding mismatch or binary corruptionPrint the minimum and maximum byte in the quality linesConvert offset or discard the file
Record count is not a multiple of fourTruncation or concatenation of mixed inputsawk 'END{print NR%4}'Recover the intact prefix or re-download
Works in one tool, fails in anotherParser strictness differsRun both parsers on the same 1000-record subsetRewrite through a strict normalizer

The Causes, From Most Common to Rarest

1. The file is not FASTQ at all

This is the single most frequent cause. A pipeline step is handed a FASTA file, a SAM file, a BAM file that was renamed to .fastq, or a plain text error log that a failed job wrote where an output file was expected. The error message mentions the header because the parser reads line 1 and finds > or @HD instead of @ plus an identifier.

BAM files deserve a specific note. They are binary and compressed, so a text editor will show unreadable bytes rather than a bad header. If your file starts with the bytes BAM\1 after decompression, or \x1f\x8b before it, you have a BAM or a gzip stream and no amount of header editing will help. Convert with samtools fastq and continue from there.

The confirmation step is trivial. head -1 reads.fastq shows the first byte. file reads.fastq distinguishes text from binary and reports gzip compression.

2. Truncated downloads and interrupted writes

A gzip stream that ends early produces a decompression error at the point of truncation. If the truncation lands inside a record, the parser sees a header without a matching sequence, or a quality line that is shorter than its sequence. If the truncation lands between records, you get a clean but incomplete file and the record count is still a multiple of four, which is why truncation is not always obvious.

Checksums are the only reliable check. Download the provider's MD5 or SHA-256 value and compare it. If no checksum is published, gzip -t confirms that the compressed stream is internally consistent, which catches the majority of incomplete transfers. Note that a file can be complete at the gzip level and still be short at the logical level if a producer wrote a partial file and then compressed it.

Interrupted writes behave the same way. A job killed during compression leaves a file that decompresses to a prefix. Workflow managers that use atomic rename or temporary directories avoid this class of error.

3. Carriage returns and Windows line endings

A FASTQ file written on Windows, or passed through a text-mode FTP transfer, can carry a carriage return before every newline. The parser reads the header line as @READ1\r and the identifier as READ1\r. The carriage return is invisible in most terminals, so the file looks correct.

The damage extends beyond the header. The sequence line gains a \r, which changes its length by one. The quality line gains a \r, which changes its length by one. Now sequence and quality are both off, and some parsers will report a length mismatch while others will accept the record and silently shift the coordinates.

There is one saving grace. Valid Phred+33 quality characters occupy ASCII 33 to 126. Carriage return is ASCII 13, which is outside that range. Stripping every \r from a FASTQ file is therefore safe and cannot destroy legitimate quality information. This is not true for arbitrary text files, and it is worth knowing why it is true here.

Confirm with file reads.fastq, which reports "with CRLF line terminators" on most builds, or with grep -c $'\r' reads.fastq under bash.

4. Wrapped records

The canonical FASTQ record is four lines. Some writers, particularly older converters and some assembler outputs, wrap long sequences across multiple lines, mimicking FASTA. A strict positional parser reads line 3 as the plus line and finds sequence data, or reads line 4 as the quality line and finds the second half of the sequence.

Biopython's SeqIO parser and a few other tolerant parsers handle wrapped records correctly, which creates a confusing situation. A file passes one tool and fails another. The difference is not the file's validity in any absolute sense, it is the parser's strictness.

The fix is to rewrite the file through a parser that knows the format, so the output is canonical four-line FASTQ. This is described in the repair section below.

5. Sequence and quality strings of unequal length

The format requires one quality character per base. When the lengths differ, the record is malformed, and every downstream consumer will either reject it or compute nonsense. Causes include wrapping, a shifted line from an earlier concatenation, a truncation that removed the tail of the quality line, and hand editing.

The check is a single pass with awk and it should be part of every ingest gate.

awk 'NR%4==2{s=length($0)} NR%4==0 && length($0)!=s{print "length mismatch at record " int(NR/4)}' reads.fastq

Records with mismatched lengths cannot be repaired in general, because there is no way to know which quality character belongs to which base. Discard the affected records and record how many were lost. If the count is more than a handful, go back to the source rather than accepting the loss.

6. A plus line that does not match the header

The plus line may repeat the header or may be bare. Both are legal. Problems arise when the plus line is truncated, when it contains a stray description that a strict parser tries to match against the header, or when the header itself was edited without editing the mirrored plus line.

Most modern parsers ignore the plus line content entirely, which is why this cause is less common than it used to be. Older tools, particularly some Java implementations and some legacy pipelines, compare the two and fail when they differ. If you have rebuilt a header, rebuild the plus line at the same time or write a bare +.

7. Duplicate, missing, or non-unique identifiers

Merging two libraries that were both named from the same instrument coordinates produces two records with the same identifier. Merging lanes is the classic case, and so is concatenating technical replicates. Some tools tolerate duplicate identifiers and some treat them as a fatal error, because downstream steps assume that a read name uniquely identifies a read.

There is a subtler failure. Many tools strip the description after the first space and keep only the identifier. If two records share an identifier but differ in the description, the tool sees duplicates that the raw file did not appear to contain. This happens routinely when the read number is stored in the space-delimited block rather than in the identifier.

Confirm with a sorted duplicate check.

awk 'NR%4==1{print substr($0,2)}' reads.fastq | sort | uniq -d | head

That command loads the identifier column into memory, which is fine for a test subset and expensive for a full flowcell. Use seqkit stats or a streaming hash counter for production volumes.

8. Whitespace and control characters inside the identifier

Tabs, non-breaking spaces, null bytes, and stray Unicode punctuation all break parsers. A tab in the header is the most common because some spreadsheet exports and some metadata joins insert them. Most parsers split the header on the first space and treat everything after it as a description, so a tab behaves unpredictably and differs between implementations.

Confirm by scanning for bytes outside the printable ASCII range.

LC_ALL=C grep -n '[^ -~]' reads.fastq | head

That pattern matches anything outside the space-to-tilde range, which includes tabs and every byte above 126. Fix by replacing the offending character with a space or by discarding the description entirely.

9. Quality encoding mismatch

A file written with offset 64 read as offset 33 produces quality scores that are all 31 points too high, plus the reverse. Some of those shifted values land on characters that are outside the valid range, which produces an "invalid quality character" error. Others land inside the range and produce a file that parses cleanly with wrong scores, which is worse.

Illumina's early pipelines used offset 64, and the Solexa variant used a different scale entirely. Modern data is almost always offset 33. If you are working with archived data from before roughly 2011, the encoding is worth confirming before you trust any quality-based filter.

A quick check: compute the minimum quality byte in a sample of records. If the minimum is 64 or above, offset 64 is likely. If the minimum is 33, offset 33 is likely. If the minimum falls between 34 and 63, the file is probably offset 33 with no very low scores, or it is corrupted.

10. Header dialect mismatch

A parser that requires the Casava 1.8 layout will reject or mis-handle a Casava 1.7 header or an SRA-style accession. The failure often appears not as a header error but as a downstream error, for example a tool that cannot determine read orientation, or a tool that produces an empty read group and then fails validation.

Confirm by looking at the first header of the file and comparing it against the layout your tool documents. Normalize at ingest rather than patching each tool.

11. Interleaved and paired-end confusion

An interleaved FASTQ holds read 1 and read 2 in the same file, alternating. A tool given an interleaved file where it expects two separate files will usually read the second record as a new header and produce a cascade of errors. The reverse also happens: a tool given two files where it expects interleaved data will treat the first record of the second file as a continuation.

Confirm by counting records. An interleaved file for a paired run has an even number of records, and the read block in the first two headers will show 1 and 2. If your two files have equal record counts and matching identifiers in the same order, they are separate mates, not interleaved.

12. Concatenation mistakes

Concatenating gzip streams is legitimate, because gzip supports multiple members in one file. cat lane1.fastq.gz lane2.fastq.gz > all.fastq.gz produces a file that decompresses to the concatenation of both. The mistake is mixing compressed and uncompressed inputs, which produces a file that decompresses into binary garbage partway through. The other mistake is concatenating bgzip blocks and indexing the result without rebuilding the index, which produces an index that points to the wrong offsets.

The safest concatenation is to decompress, concatenate, and recompress in a single stream.

cat lane1.fastq lane2.fastq | pigz -p 8 > all.fastq.gz

A Diagnostic Sequence You Can Run in Order

Run these steps in order and stop at the first failure. Each step is cheap relative to the cost of a failed alignment job.

  1. Confirm the file type and compression.
file reads.fastq.gz
  1. Confirm that the compressed stream is intact.
gzip -t reads.fastq.gz
  1. Read the first two records.
zcat reads.fastq.gz | head -8
  1. Count lines and check the remainder modulo four.
zcat reads.fastq.gz | awk 'END{print "lines=" NR " remainder=" NR%4}'
  1. Confirm that every first-of-four line starts with @.
zcat reads.fastq.gz | awk 'NR%4==1 && substr($0,1,1)!="@"{print "bad header at line " NR; exit}'
  1. Confirm that every third-of-four line starts with +.
zcat reads.fastq.gz | awk 'NR%4==3 && substr($0,1,1)!="+"{print "bad plus at line " NR; exit}'
  1. Compare sequence and quality lengths.
zcat reads.fastq.gz | awk 'NR%4==2{s=length($0)} NR%4==0 && length($0)!=s{print "length mismatch at record " int(NR/4); exit}'
  1. Check the quality byte range.
zcat reads.fastq.gz | LC_ALL=C awk 'NR%4==0{for(i=1;i<=length($0);i++){c=substr($0,i,1); if(c<"!" || c>"~"){print "bad quality char at record " int(NR/4); exit}}}'
  1. Get a structural summary from a dedicated tool.
seqkit stats -a reads.fastq.gz

That prints format, number of sequences, total length, minimum and maximum length, and quality statistics. A file that seqkit cannot parse has a real defect.

  1. Run a quality control pass on a subset.
zcat reads.fastq.gz | head -400000 | fastqc stdin --outdir qc_subset

FastQC reports a sequence and quality length mismatch as a hard error rather than producing a report, which is a fast confirmation that the records are structurally broken.

Do not skip step 2. A truncated gzip stream often produces a parse error halfway through a large file, and the error message points at the record where decompression stopped rather than where the transfer failed.

Fixes That Preserve the Data

Every repair in this section writes to a new file. The original stays untouched until the repaired file has passed the diagnostic sequence above.

Re-download and verify

If the checksum does not match, no repair is appropriate. Re-download and verify again. Retrying a transfer with a resume flag on a server that does not support it can produce a file that is the right size and still corrupt, which is why the checksum matters more than the file size.

md5sum reads.fastq.gz
sha256sum reads.fastq.gz

Normalize line endings

tr -d '\r' < reads.fastq > reads_lf.fastq

dos2unix reads.fastq achieves the same result and is available in most package managers. Both are safe for FASTQ because carriage return is not a valid quality character.

Unwrap records

The most reliable path is a parser that understands the format and writes canonical four-line output. Biopython's SeqIO handles wrapped input and emits normalized records.

python - <<'PY'
from Bio import SeqIO
with open("wrapped.fastq") as handle, open("flat.fastq", "w") as out:
    for record in SeqIO.parse(handle, "fastq"):
        quality = "".join(chr(q + 33) for q in record.phred_quality)
        out.write("@%s\n%s\n+\n%s\n" % (record.description, record.seq, quality))
PY

That snippet also normalizes the plus line to a bare + and re-emits quality in Phred+33. If the source was Phred+64, the conversion inside Biopython will already have accounted for it only if Biopython detected the encoding correctly, so verify the encoding independently for archived data.

seqkit seq -w 0 is a faster alternative for large files when the input parses cleanly.

Rebuild and normalize headers

If the identifiers need a prefix, or the description needs to be stripped, rewrite only the header lines. The trap here is a naive substitution.

sed 's/^@/@sampleA_/' reads.fastq > prefixed.fastq

That command looks correct and is wrong. It rewrites every line that begins with @, including quality lines where @ encodes Q31, which is a common score in real data. The result is a file with corrupted quality strings and a header error that appears somewhere else entirely.

Restrict the substitution to header positions.

awk 'NR%4==1{sub(/^@/,"@sampleA_")} {print}' reads.fastq > prefixed.fastq

To strip everything after the identifier and keep only the first token:

awk 'NR%4==1{sub(/ .*/,"")} {print}' reads.fastq > id_only.fastq

That normalizes every dialect to a bare identifier, which resolves most dialect mismatches at the cost of losing the read number, the filter flag, and the index sequence. If a downstream tool needs the read number, write it back explicitly as /1 and /2 from the file's mate designation rather than discarding it.

Resolve duplicates

Prefix each input with a distinct sample tag before merging. That is cheaper and more reversible than deduplicating identifiers after the fact.

awk 'NR%4==1{sub(/^@/,"@lane1_")} {print}' lane1.fastq > lane1_tagged.fastq
awk 'NR%4==1{sub(/^@/,"@lane2_")} {print}' lane2.fastq > lane2_tagged.fastq
cat lane1_tagged.fastq lane2_tagged.fastq | pigz -p 8 > merged.fastq.gz

If tagging is not possible, add a numeric suffix on collision. The streaming approach keeps memory bounded.

zcat merged.fastq.gz | awk 'NR%4==1{id=substr($0,2); n[id]++; if(n[id]>1){sub(/^@/,"@" id "_dup" n[id])}} {print}' | pigz -p 8 > deduped.fastq.gz

Repair or discard length mismatches

There is no honest repair. A record whose quality string is shorter than its sequence has lost information. Filter it out and count the loss.

awk 'NR%4==2{s=length($0)} NR%4==0 && length($0)==s{print prev2 "\n" prev1 "\n" prev3 "\n" $0} {prev3=prev2; prev2=prev1; prev1=$0}' reads.fastq > good.fastq

That buffer-based filter is awkward. A cleaner route is to let a tool that already validates records do the filtering, or to use the seqkit statistics to confirm the problem before writing a custom filter.

If the mismatch comes from wrapping, unwrap first. Unwrapping resolves most length mismatches because the quality string was never actually short, it was just split.

Convert quality encodings

Converting offset 64 to offset 33 means subtracting 31 from every quality byte. The inverse means adding 31. Both operations are simple and both are destructive if applied twice, so record in the file name or the pipeline metadata which conversion was applied.

python - <<'PY'
with open("in_phred64.fastq") as fin, open("out_phred33.fastq", "w") as fout:
    while True:
        header = fin.readline()
        if not header:
            break
        seq = fin.readline().rstrip("\n")
        plus = fin.readline()
        qual = fin.readline().rstrip("\n")
        fout.write(header)
        fout.write(seq + "\n")
        fout.write(plus)
        fout.write("".join(chr(ord(c) - 31) for c in qual) + "\n")
PY

After conversion, recheck the minimum quality byte. It should be 33 or higher, and it should not be 64 or higher across a large sample unless the data genuinely has no low-quality bases, which is rare.

Rewrite through a strict normalizer

When several defects coexist, the fastest path is to push the records through a strict parser that emits canonical output and to discard anything it cannot parse. That converts an unknown number of subtle defects into a known number of lost records, which is a trade you can defend. Record the input count, the output count, and the difference in the run log, and treat a large difference as a reason to go back to the source.

Safety Note

Several commands in this guide write files and one of them can truncate your input. Shell redirection with > opens the target before the command runs, so a command like zcat reads.fastq.gz > reads.fastq.gz destroys the input before it can be read. Always write to a new name, always keep the raw file read-only, and check the target path before you press enter.

Deleting or overwriting raw sequencing data is irreversible. Set the raw directory to read-only and do all work in a scratch directory.

Sequence data from human subjects is protected health information in most US research contexts. Do not upload raw FASTQ files to public tools or paste header lines into public forums without checking your institution's data use agreement, because headers can contain flowcell identifiers and index sequences that link back to a specific run.

Prevention Practices

Validate at ingest

Put the diagnostic sequence at the top of the pipeline, before any analysis. A single validation task that checks the format, the plus line, the length agreement, and the record count modulo four catches the overwhelming majority of defects in seconds. Fail the run rather than passing a malformed file downstream.

Keep raw data immutable

Set the raw directory to read-only, compute a checksum on arrival, and store the checksum alongside the file. Every repair writes a new file. Every downstream step reads from the validated copy.

Normalize headers once, at the boundary

Pick a canonical header form for your project, for example the bare identifier with a sample prefix, and apply it as the first transformation after validation. Store the mapping from original to canonical identifier so that you can trace any read back to the instrument. Tools that need the full Casava layout should receive it from the original, not from the normalized copy.

Name mates consistently

Use _R1 and _R2 or _1 and _2 in file names, and keep the order identical across all samples. Most paired-end errors are name-matching errors, not format errors.

Pin tool versions and record them

Header parsing behavior changes between tool versions. A pipeline that works today can fail after an update. Pin versions in the workflow file and write them into the run manifest.

Test repairs on a subset

Before you rewrite a 200 GB file, run the repair on the first 100,000 records and validate the output. The failure mode of a bad repair is a file that parses and is wrong, which is much harder to catch than a file that fails to parse.

Compress with intent

Use pigz for speed on general FASTQ and bgzip only when you need random access and indexing. Do not mix the two in one concatenated file.

Common Mistakes and Limitations

Counting @ characters to count records is wrong. In Phred+33, @ is Q31 and + is Q10, both common quality scores, so grep -c '^@' and grep -c '^+' both overcount. Count positionally with awk 'NR%4==1' or use a real parser.

Running wc -l on a gzipped file returns the number of newline bytes in the compressed stream, not the number of records. Always decompress first or use a tool that understands compression.

Trusting a tool that accepts the file is not validation. Tolerant parsers accept wrapped records, tab-separated descriptions, and offset-64 qualities. Strict parsers reject them. The fact that one tool accepted the file tells you about that tool, not about the file.

Editing FASTQ with line-oriented text tools is risky for the same reason. Any pattern that can match a quality character can corrupt quality data. Restrict substitutions to NR%4==1 and NR%4==3.

Changing a file extension does not convert a format. A FASTA file renamed to .fastq is still FASTA, and a BAM file renamed to .fastq is still binary.

Assuming Phred+33 for all archived data is a mistake. Confirm the encoding on any dataset older than about 2011 before you trust a quality filter.

FASTQ header content has no single normative specification. The format description establishes the four-line structure and the quality encoding, and the header schema is a platform convention. This is a genuine limitation, not a gap in your knowledge. Two files that both parse can carry identifiers with entirely different semantics, and a tool that works on one will silently misread the other.

Validation cannot detect sample swaps, index hopping, or a reversed mate assignment. Those are wet-lab and demultiplexing problems that a format check will never find. Pair them with a genotype concordance check or a fingerprinting step when sample identity matters.

Individual files that resist every repair in this guide need to be traced back to the producing instrument or pipeline. Do not invent quality scores, do not interpolate missing bases, and do not hand-edit records to make a parser accept them.

Frequently Asked Questions

What is the most common cause of a FASTQ header format error?

The file is not FASTQ, or it is a truncated copy of one. Check the first byte of the first line and verify the gzip stream before you look anywhere else.

How can I tell whether a file is FASTQ or FASTA?

Look at the first character. FASTQ headers begin with @ and each record has four lines. FASTA headers begin with > and sequences may wrap across any number of lines.

Can a FASTQ record span more than four lines?

Yes, some writers wrap long sequences, and some parsers accept it. The canonical form is four lines, and strict parsers reject wrapped records. Rewrite wrapped files through a normalizing parser before analysis.

Why does a file pass FastQC but fail in my aligner?

FastQC is tolerant of some header dialects and reports quality metrics rather than enforcing a schema. Aligners often need a specific read-number convention or a read group. Normalize headers at ingest so both tools see the same layout.

How do I fix duplicate read identifiers in a FASTQ file?

Prefix each source file with a unique sample tag on the header line only, then merge. Do the substitution with a positional rule such as awk 'NR%4==1' so quality lines are never touched.

What is the difference between Phred+33 and Phred+64 quality encoding?

They differ by an offset of 31. Phred+33 stores Q0 as ! and is the modern standard. Phred+64 stores Q0 as @ and appears in archived Illumina data. Reading one as the other shifts every score by 31 points.

How do I remove Windows line endings from a FASTQ file?

Strip carriage returns with tr -d '\r'. This is safe for FASTQ because carriage return is not a valid quality character, so the operation cannot delete real quality data.

Can I convert a FASTA file to FASTQ?

Only by inventing quality scores, which is almost always the wrong choice. Most tools that accept FASTQ have a documented FASTA mode, and fabricated qualities distort every downstream quality filter.