How to Diagnose and Fix Truncated FASTA and FASTQ Files

By Dr. Zubair Khalid, DVM, MS, PhD · 2026-11-10

A truncated FASTA or FASTQ file is almost always the fingerprint of a write that stopped early. The most common single cause is an interrupted transfer: a download that lost its connection, an scp that was killed, a browser save that timed out, or a pipeline that ran out of disk space while streaming reads to a compressed output. The first thing to check is not the sequence content. It is the byte count and the record boundary.

Two questions resolve most cases in under a minute. Does the file end at a clean record boundary? And does it match the size or checksum published by whoever produced it?

If the answer to either question is no, you have a truncated file. The rest of the work is deciding whether the complete records you already have are worth keeping, whether the missing tail can be re-fetched, or whether the run has to be repeated. That decision depends on the format, because FASTQ and FASTA give you very different amounts of built-in protection.

This guide covers both formats, works from the most common cause to the rarest, gives commands you can run immediately, and ends with the ingest controls that stop truncated files from entering an analysis in the first place.

FASTQ vs FASTA: Why One Fails Loudly and the Other Fails Silently

Both formats are plain text. Both describe biological sequences. The difference in how they fail under truncation is structural, not biological, and it drives every diagnostic decision that follows.

The FASTQ record has a built-in length check

A FASTQ record occupies exactly four lines:

1. A header line that begins with @, containing the read identifier and often a space-separated description.
2. The sequence line, using IUPAC nucleotide codes (ACGT plus N and ambiguity codes).
3. A separator line that begins with +, optionally repeating the header.
4. A quality line using one ASCII character per base.

The format was developed at the Wellcome Trust Sanger Institute to carry per-base quality scores alongside each read, and it remains the standard output of Illumina, Element, PacBio, and Oxford Nanopore instruments after basecalling.

The critical invariant is that line 2 and line 4 must have identical length. That single rule gives you a free integrity test on every record. A cut in the middle of a quality line produces a length mismatch. A cut in the middle of a sequence line produces a mismatch too, because the quality line that follows will be full length. A cut in the middle of a header line leaves a line that does not begin with @. Only a cut exactly at a record boundary is invisible to a naive line count, and that case is caught by the divisibility check described below.

Quality characters encode error probabilities. Modern Illumina data uses Phred+33, where the character ! (ASCII 33) means Q0 and I (ASCII 73) means Q40. Legacy Illumina 1.3 to 1.7 data used Phred+64, and mixing the two offsets is a separate class of bug that truncated files can mimic by removing the low-quality reads that would have made the offset obvious.

The FASTA record has no length check at all

A FASTA record is a header line beginning with > followed by one or more sequence lines. That is the entire specification. There is no declared length, no quality string, and no terminator. Format conventions allow line wrapping at 60 or 70 or 80 characters, and the parser is expected to concatenate the sequence lines and strip whitespace.

This is why a truncated FASTA file is dangerous. If the write stops in the middle of record 4,000, you get a valid FASTA file containing 3,999 complete records and one record that is short by an unknown number of bases. Every parser will read it without complaint. The file will assemble, align, or translate, and the only symptom is a missing gene, a broken open reading frame, or an assembly that collapses at one locus. Diagnosis requires an external reference: a checksum from the archive, a record count from a manifest, or a declared length inside the record.

That last option exists in some archives. UniProt flat files carry a SQ line that states the residue count explicitly, and GenBank records carry a LOCUS line with a base count. A plain FASTA download does not, which is why checksum verification matters more for FASTA than for FASTQ.

What this means in practice

For FASTQ, a single awk pass can locate the exact line where integrity ends. For FASTA, you need an expectation to compare against. Plan your validation accordingly.

Where Truncation Comes From

Truncation is a write-side failure. Something that should have written N bytes wrote fewer, and the filesystem recorded what it got. The producer may not have noticed, and the consumer almost never does.

Five mechanisms account for nearly all real-world cases:

• The transfer ended before all bytes arrived.
• The compression stream ended before its final block and trailer.
• The storage ran out of space, quota, or inodes.
• The writing process was killed, preempted, or lost its host.
• A pipeline stage failed, and the failure was not propagated.

Two more mechanisms matter for smaller files: text-handling damage from editors and email, and tool-level errors during format conversion. They are rarer but produce much more confusing symptoms because the file size can look reasonable.

The First Five Minutes: A Diagnostic Sequence

Run these in order. Each step is fast and each one narrows the search.

1. Print the size and the last byte. ls -l reads.fq.gz genome.fa and tail -c 1 genome.fa | od -An -c. A file that does not end in a newline character is a candidate for truncation, even though many parsers tolerate it.

1. Test the compression container. gzip -t reads.fq.gz returns 0 on success. Any nonzero exit, especially with unexpected end of file, means the compressed stream is incomplete. Use pigz -t for large files because it tests in parallel.

1. Check FASTQ line count divisibility. zcat reads.fq.gz | wc -l must be divisible by 4 for a single-end file and by 8 for a correctly interleaved paired-end file. A remainder of 1, 2, or 3 is conclusive evidence of truncation.

1. Check record-internal lengths. Run the awk validator shown in the repair section below. It flags header, separator, and quality-length errors line by line.

1. Compare against the source. Fetch the published checksum file and run sha256sum -c or md5sum -c. For NCBI SRA downloads, vdb-validate SRR12345678 verifies the local copy against the archive. For release directories at NCBI and EMBL-EBI, per-file checksums are published next to the data.

1. Check the environment for the cause. df -h, df -i, quota -s, and on Linux dmesg | grep -i "killed process" or journalctl -k | grep -i oom. A full disk explains the symptom, and it also explains why the same pipeline produced a good file last week.

Troubleshooting Table: Symptom, Cause, Confirmation, Fix

Cause by Cause: Diagnosis and Repair

1. An interrupted transfer (the most common cause)

A download that stalls at 94 percent leaves a file whose name is correct and whose content is not. The failure modes are mundane: a laptop went to sleep, a VPN dropped, a firewall reset an idle connection, a shared cluster login node rebooted, or a browser timed out on a 40 GB file.

The tell is that the file size is smaller than the published size, usually by a large and non-round amount. Compare ls -l output to the size listed in the archive's directory listing or metadata record.

Fix this by re-fetching with resume support rather than starting over:

wget -c https://example.org/release/reads.fastq.gz

curl -C - -O https://example.org/release/reads.fastq.gz

rsync --partial --progress --append-verify user@host:/data/reads.fastq.gz ./

wget -c and curl -C - send a range request and append the remaining bytes. rsync, partial keeps a partial file if the transfer dies, and -append-verify checks the assembled file once it is complete. Neither option is a substitute for checksum verification, because a resume can splice bytes from two different versions of a file if the source changed between attempts.

The definitive fix is the checksum. Download the companion checksum file and verify:

sha256sum -c reads.fastq.gz.sha256

If the source does not publish a checksum, record the size and the SHA-256 of every file at the moment you receive it, and treat that record as your baseline for future comparisons.

Safety note: never redirect a decompression into the compressed file itself. The shell truncates the target before the reader opens it, and the source data is gone. gzip -dc reads.fq.gz > reads.fq.gz destroys the file.

2. A gzip stream that ends early

Gzip wraps the original data in a series of deflate blocks and finishes with an 8-byte trailer holding a CRC32 and the uncompressed size in modulo 2^32 form. If the file stops before that trailer, it is incomplete, and gzip -t will say so.

The trap is that gzip -dc and zcat will happily write out every complete block they find and then exit with code 1 and a message on standard error. If your pipeline calls zcat file.gz | tool without set -o pipefail, the tool may receive 60 percent of your reads, complete successfully, and produce a result nobody questions.

Confirm the failure:

gzip -t reads.fq.gz
echo $?

For very large files, pigz -t is much faster because it decompresses in parallel threads:

pigz -t reads.fq.gz

For other containers the equivalent tests are bzip2 -t, xz -t, zstd -t, and lz4 -t. All of them return nonzero on truncation.

Fix a truncated gzip by decompressing the salvageable prefix and then checking the record structure:

zcat reads.fq.gz > reads.partial.fq || echo "stream ended early"

Then run the FASTQ validator and cut at the last complete record as described below. Do not simply gunzip the prefix and call it done, because the last record will often be half-written.

BGZF files, the blocked gzip variant used inside BAM and some FASTQ distributions, behave differently. Each block is independently decompressible, so a truncated BGZF file loses only the final block. BAM adds a dedicated 28-byte empty block as an end-of-file marker, which is why samtools quickcheck can report that a BAM "was not terminated by an EOF block". Use that command before any downstream BAM processing.

3. Disk full, quota exceeded, or inodes exhausted

A writer that hits ENOSPC gets a failed write() call. Whether that becomes a clean error or a silently truncated file depends entirely on whether the program checks the return value. Many do not check it correctly on the final buffered flush, so the process can exit 0 while the last few megabytes never reached storage.

Three distinct limits cause this:

• Block space. df -h shows it. The classic case is a /tmp or $TMPDIR that is smaller than the output, which matters for fasterq-dump and for sort-based tools that write intermediates.
• Inodes. df -i shows it. A directory with millions of tiny per-barcode FASTQ files can exhaust inodes long before it exhausts bytes.
• User or project quota. quota -s on systems that enforce it. Cross that limit and writes fail at a size that looks arbitrary.

Diagnose by correlating the file's final size with the timestamp of the failure. If the session log shows an out-of-space warning at the same minute the file stopped growing, the cause is settled.

Fix by freeing space or relocating the output, then re-running the writing step from the last checkpoint. Re-running only the final stage is much cheaper than repeating an entire sequencing run or basecall. If the writer has no checkpoint, the output must be regenerated from the primary data.

Prevent recurrence by reserving headroom. A common convention is to keep at least 20 percent of the target volume free, and more for sort-heavy steps. Some workflow managers can be configured to check free space before launching a task.

4. A killed or preempted writer

Long-running producers die from causes that have nothing to do with space. The Linux OOM killer sends SIGKILL when memory pressure spikes. A Slurm or Kubernetes job hits its walltime and is preempted mid-write. A container is evicted. A cloud spot instance is reclaimed. A workstation reboots during a basecall that had been running for 30 hours.

SIGKILL is a specific hazard because the process cannot flush buffers, close files, or write a termination marker. The result is the same as a power cut at the filesystem level: everything still in the page cache may survive, but nothing after it does.

Confirm the kill in the system logs:

dmesg | grep -i "killed process"
journalctl -k | grep -i oom
sacct -j 1234567 --format=JobID,State,ExitCode,MaxRSS,Elapsed

For sequencing runs specifically, check the instrument's own run metadata before blaming the transfer. Illumina run folders declare the number of cycles in RunInfo.xml. If the run stopped early, the FASTQ files will be internally consistent but shorter in cycle count than the recipe specified. That is a different problem from truncation, and re-downloading will not fix it.

Fix by regenerating the output from whatever survives. For basecalling, that means re-running the basecaller against the raw signal data (BCL, POD5, or the vendor's native format). For a conversion step, it means re-running the conversion on intact input. Fixing the FASTQ itself is not possible because the missing bases were never written.

5. Pipeline breakage and SIGPIPE

Shell pipelines lose information by design. By default, the exit status of a | b | c is the status of c only. If a dies, b and c may exit 0 after processing a partial stream.

The most common self-inflicted version is a stray head. Command zcat huge.fq.gz | head -n 4000000 > subset.fq produces a truncated gzip input stream, a SIGPIPE in zcat (exit code 141), and a perfectly clean-looking output file. Nobody notices because the output was supposed to be a subset.

The other common version is a producer that fails quietly: samtools fastq on a truncated BAM, fasterq-dump that lost its temp space, or a Python script that caught an exception and exited 0.

Fix the pipeline first. Add these two lines to the top of every analysis script:

set -euo pipefail

pipefail makes the pipeline return the first nonzero status among its members. -e stops the script on any unhandled failure. -u catches unset variables, which prevents an empty variable from becoming an empty output path.

Then write outputs atomically. Write to a temporary name in the same directory and rename on success:

zcat reads.fq.gz > reads.fq.tmp && mv reads.fq.tmp reads.fq

mv within a filesystem is atomic. A downstream job that sees reads.fq sees a complete file, or sees nothing at all. This single habit prevents most accidental truncation from reaching an analysis.

6. Editor, email, and line-ending damage

Small FASTA files carry a unique risk: they get moved by hand. A sequence copied out of a web page, pasted into an email, forwarded through a client that wraps lines at 78 characters, and saved from a word processor can arrive with smart quotes, non-breaking spaces, and a hard length limit imposed by the mail system.

Symptoms include:

• A header that starts with a curly quote instead of >.
• Sequence lines containing U+00A0 non-breaking spaces that look identical to normal spaces.
• Line endings converted to CRLF by an FTP client in ASCII mode or by a Windows editor.
• Line endings converted to a bare CR, which makes wc -l report zero or one.
• Records silently dropped because the mail gateway truncated the attachment.

Diagnose with file and a CR scan:

file genome.fa
grep -c $'\r' genome.fa

Fix with a normalization pass:

dos2unix genome.fa
sed -i 's/\r$//' genome.fa

Then re-check that the first character is > and that no non-ASCII bytes remain:

LC_ALL=C grep -n '[^ -~]' genome.fa | head

Safety note: do not open FASTA or FASTQ files in a spreadsheet. Excel will reformat numeric-looking accessions, strip leading zeros, convert some values to dates, and silently change the file on save. If you need to inspect a table, use a text editor that does not reinterpret content, or convert to a table format first.

The permanent fix for hand-carried files is a checksum. Send the SHA-256 alongside the file and compare after receipt. A one-line message prevents an afternoon of debugging.

7. Truncation created by tools

The rarest group is damage introduced by a program that was supposed to process the file cleanly.

Real examples:

• fasterq-dump running out of temporary space, leaving a partial FASTQ and a nonzero exit that was ignored.
• An old 32-bit binary hitting a 2 GB file size limit.
• A converter that assumes Unix line endings and stops at the first \r.
• A tool that writes output as it reads, so an input read error becomes an output truncation.
• A network filesystem client losing its cached writes when the mount drops.

Diagnose by re-running the exact command on a small, known-good input. If the small input produces a correct output, the tool is fine and the problem is input size, disk, or the filesystem. If it fails on good input, the tool or its environment is the cause.

Fix by upgrading the tool, changing the output location to local storage, or splitting the input into chunks that are processed independently and concatenated with a validated method. Chunking also limits blast radius: a failure costs one chunk, not the whole file.

Repairing a Truncated FASTQ at the Last Complete Record

This is the workhorse repair. It keeps every intact record and drops only the incomplete tail. Log the number of lost records, because that number belongs in your QC report.

First, validate and report the last good line number:

zcat reads.fq.gz | awk '
  NR%4==1 && $0 !~ /^@/ {print "bad header at line " NR > "/dev/stderr"; exit}
  NR%4==2 {s = length($0)}
  NR%4==3 && $0 !~ /^\+/ {print "bad separator at line " NR > "/dev/stderr"; exit}
  NR%4==0 {
    if (length($0) != s) {print "quality length mismatch at line " NR > "/dev/stderr"; exit}
    last = NR
  }
  END {print last + 0}
'

The script prints the last validated line to standard output and the reason for any failure to standard error. Then keep exactly that many lines:

n=$(zcat reads.fq.gz | awk 'NR%4==1 && $0 !~ /^@/{exit} NR%4==2{s=length($0)} NR%4==3 && $0 !~ /^\+/{exit} NR%4==0{if(length($0)!=s)exit; last=NR} END{print last+0}')
echo "keeping $((n/4)) records"
zcat reads.fq.gz | head -n "$n" > reads.repaired.fq

gzip -c reads.repaired.fq > reads.repaired.fq.gz

Notes on this recipe:

• head -n on the decompressed stream sends SIGPIPE to zcat, which is expected and harmless. Do not add pipefail to the inner command substitution.
• If the file is interleaved paired-end, confirm the repaired record count is even. An odd count means the cut landed between a forward and a reverse read.
• If the input is a directory of per-sample files, apply the same recipe to each file and record the loss per sample. A single sample losing 50 percent of its reads while others lose none points to a per-file write problem, not a shared cause.
• If the loss is larger than a few thousand reads, re-download instead of repairing. Repair is a salvage operation for small tails, not a substitute for obtaining the full dataset.

For badly malformed files where line structure itself is broken, the BBTools suite includes repair.sh, which is built to salvage truncated and corrupt FASTQ and to re-synchronize paired files whose records no longer line up. Run repair.sh with no arguments to print the option set for your installed version.

Biopython can also salvage. SeqIO.parse is strict, but FastqGeneralIterator from Bio.SeqIO.QualityIO is not:

import gzip
from Bio.SeqIO.QualityIO import FastqGeneralIterator

kept = 0
with gzip.open("reads.fq.gz", "rt") as handle, open("rescued.fq", "w") as out:
    try:
        for title, seq, qual in FastqGeneralIterator(handle):
            if len(seq) != len(qual):
                break
            out.write(f"@{title}\n{seq}\n+\n{qual}\n")
            kept += 1
    except Exception as exc:
        print("stopped with", exc)
print("kept", kept)

This approach reads the file once, never loads it into memory, and stops at the first inconsistency.

How to Fix a Truncated FASTA File: Salvage, Re-Fetch, or Re-Run

FASTA repair requires an external expectation. Decide which expectation you have before touching the file.

Case 1: You have a checksum from the source. Re-download and verify. Do not spend time repairing. A checksum mismatch cannot tell you where the file broke, and a partial repair leaves a file that fails the checksum again.

Case 2: You have a record count from a manifest. Count your records and compare:

grep -c '^>' genome.fa
seqkit stats -a genome.fa

If your count is one short and the last record is clearly cut off, drop it. If the count matches but a gene is missing, the file was truncated and re-assembled or the wrong file version was downloaded.

Case 3: You have no expectation at all. Build one from the data itself. Print the length of every record and inspect the last few:

awk '/^>/{if(n)print name"\t"len; name=substr($0,2); len=0; n++; next}{len+=length($0)} END{print name"\t"len}' genome.fa | tail -n 5

A final record that is dramatically shorter than the rest, or one whose length is not a multiple of three for a coding sequence, is suspicious. For coding regions, a length that does not divide by three is a strong signal of truncation, provided the record is a single complete open reading frame rather than a multi-exon genomic sequence.

Case 4: You can re-fetch individual records. This is often the fastest path. Pull the specific accession from the source archive rather than the whole release.

curl "https://rest.uniprot.org/uniprotkb/P12345.fasta" -o P12345.fa
curl "https://www.ebi.ac.uk/ena/browser/api/fasta/ACCESSION" -o record.fa

For NCBI records, use NCBI E-utilities efetch with rettype=fasta, or the efetch wrapper from Entrez Direct.

Then splice the record back into the assembly-level file at the correct position, or rebuild the file from a manifest of accessions. That rebuild approach is reproducible and leaves an audit trail.

Case 5: The file is a single long sequence with no per-record structure. A truncated chromosome in a reference assembly is not recoverable by any local edit. Re-download the assembly, verify against the published checksum, and check that the contig or chromosome lengths in the FASTA index match the published statistics. samtools faidx genome.fa followed by cut -f1,2 genome.fa.fai gives you every sequence name and length in one table.

A note on wrapping. FASTA line wrapping is cosmetic. If you re-wrap a file, the byte count changes and a strict byte-level checksum will no longer match even though the sequence content is identical. Verify against a sequence-level digest, not a file-level one, when comparing wrapped and unwrapped copies.

Validating the Repair

A repair is not finished until the output passes every check the input failed.

Run this checklist on every repaired file:

1. Container integrity. gzip -t returns 0.
2. Record structure. The awk validator reports no errors and the FASTQ line count is divisible by 4, or divisible by 8 if interleaved.
3. Record count. Compare seqkit stats output to the expected count minus the logged loss.
4. Length distribution. seqkit stats -a reports min, average, max, and read-length quartiles. A repaired FASTQ from an Illumina run should show a tight distribution around the instrument's configured read length. A sudden spread means partial records survived somewhere.
5. Quality range. Confirm the minimum and maximum quality characters are legal for the declared encoding. Any byte below 33 or above 126 means the quality line was damaged, not merely short.
6. Duplication and adapter signal. A FastQC run after repair should show the same general profile as before, minus the affected region. New adapter peaks or a sudden GC shift indicate a splice error.
7. Reproducibility record. Compute and store sha256sum of the repaired file alongside the repair command and the number of records lost.

For BAM inputs, add samtools quickcheck and samtools flagstat. A BAM that lost its EOF block may still be readable, but samtools and Picard treat it as suspect and downstream tools can behave inconsistently.

Preventing Truncation in the First Place

Prevention is cheaper than every repair technique in this article.

Fail loudly in every script. Put set -euo pipefail at the top of every Bash script. In Python, do not wrap the main loop in a bare except that swallows the error. In Nextflow or Snakemake, keep the default error strategy, which stops the run on a nonzero exit.

Write atomically. Write to a temporary path in the destination directory and rename on success. Never let a partially written file carry the final name.

Verify at ingest. Treat a checksum as part of the file. Download the checksum with the data and run sha256sum -c before any downstream rule fires. For SRA data, vdb-validate is the equivalent gate.

Keep an immutable raw layer. Store received data read-only (chmod -R a-w) and never edit a raw file in place. Every transformation produces a new file with a new checksum.

Publish checksums for anything you distribute. A short sha256sum *.fq.gz > MANIFEST.sha256 costs seconds and saves every downstream collaborator a day of ambiguity.

Test the container at ingest. Run gzip -t or pigz -t over every compressed file, even in a fast local pipeline. A parallelized test on a large archive takes a fraction of the time it takes to diagnose a bad merge later.

Give writers room and time. Monitor df -h and df -i before launching anything that writes more than a few gigabytes. Reserve headroom. Run long jobs under tmux or screen, or on a scheduler with checkpointing, so a dropped connection does not kill the process.

Use a workflow manager for anything multi-step. A real workflow engine propagates exit codes, records tool versions, caches intermediate files, and can re-run only the failed step. Most truncation that reaches final results is a shell pipeline that ignored a failure.

Record provenance. Keep the command, the tool version, the input checksums, and the output checksums for each step. When a downstream result looks wrong, provenance turns a speculative hunt into a comparison.

Common Mistakes and Limitations

Assuming a successful exit means a complete file. Many writers do not check the final write() return value. A tool can exit 0 with a truncated output. Verify sizes and checksums, not exit codes alone.

Truncating at a line count without checking record internals. A file cut mid-quality-line can still have a line count divisible by 4 if the cut happens to fall just before a newline. Always validate sequence and quality lengths together.

Running head on a compressed file. head on a .gz produces a partial gzip member, not a partial subset of records. Decompress first, then subset, then recompress.

Trusting wc -l when the final line lacks a newline. wc -l counts newline characters, so a file whose last line is complete but unterminated is undercounted by one. Check the last byte separately.

Deleting the original before the repair is verified. Keep the truncated file until the repaired file passes all checks. The truncated file is evidence, and sometimes it is the only copy of the reads you have.

Assuming a checksum mismatch always means truncation. A file that was recompressed at a different gzip level has the same decompressed content and a different file checksum. Use a decompressed-content digest when the source recompressed the data.

Repairing a file that should be re-downloaded. Repair is appropriate for a small tail loss. Salvaging 60 percent of a dataset and proceeding to assembly is a decision that must be documented, not hidden.

Splicing files from different runs. Concatenating two FASTQ files is valid only if they share read length, quality encoding, and adapter chemistry. Mixed encoding produces quality values that are silently wrong by 31 points.

Opening sequence files in Word, Excel, or a chat client. Any of these can change characters silently. Use scp, rsync, or a repository.

Ignoring the possibility that the middle is lost, not the end. Disk-full and killed-writer cases truncate the tail. Filesystem corruption, a bad merge, or a botched cat can remove an interior block while keeping the tail intact. Checksums catch this. Line counting does not.

Limitation of all local repair. If the bytes are gone, they are gone. No command reconstructs bases that were never written. When the missing data matters, the only real fix is to regenerate the source file or re-fetch it from the archive.

Individual datasets vary, and a repair that is safe for one experiment may be unacceptable for another. Decisions about whether a partially recovered dataset is fit for a publication-grade analysis belong with the analyst who owns the experiment.

Frequently Asked Questions

What is the fastest way to tell if a FASTQ file is truncated?

Run zcat reads.fq.gz | wc -l and check whether the result is divisible by 4. A remainder of 1, 2, or 3 confirms truncation, and the awk validator locates the exact line where structure breaks.

Can a truncated FASTA file be repaired automatically?

No, not reliably. FASTA has no declared length or terminator, so a repaired file looks valid even when the last record is short. Verify against a published checksum, a record count from a manifest, or a sequence length from the source archive before trusting any repair.

Why does gzip say "unexpected end of file"?

The gzip stream is missing its final deflate block or its 8-byte trailer, which holds a CRC32 and the uncompressed size. The file was cut short during writing or transfer, and zcat will write whatever decompressed prefix exists before exiting with a nonzero status.

Is it safe to keep the first complete records and discard the rest?

Yes, for many purposes, provided the loss is small and documented. Record the number of reads or sequences discarded, keep the truncated original, and check that the repaired file passes length-distribution checks before downstream analysis.

Why does FastQC fail on my FASTQ file?

FastQC reports unequal sequence and quality lengths, or throws an exception when it reaches a malformed record. That is the truncated tail surfacing. Validate and cut the file at the last complete record, then run FastQC again.

Can a file be truncated if ls -l shows a plausible size?

Yes. A writer that fails silently, a proxy serving a cached partial copy, or a resume that spliced two file versions can all produce a size that looks reasonable. Only a checksum comparison against the published source settles the question.

How do I stop truncated files from entering a pipeline again?

Add checksum verification and a format validation step as the first rule in the workflow. Set set -euo pipefail in shell scripts, write outputs to temporary paths and rename on success, and rely on a workflow manager that stops on nonzero exit codes.

Why does Biopython raise a ValueError when I parse my FASTQ file?

Bio.SeqIO.parse with the fastq format enforces equal sequence and quality lengths and raises when it finds a mismatch. Use FastqGeneralIterator from Bio.SeqIO.QualityIO to read records one at a time and stop cleanly at the first broken record.