How to Recover Data from a Corrupted BAM File: Tools and Techniques

By Dr. Zubair Khalid, DVM, MS, PhD · 2026-11-10

A BAM file that will not open is usually not a failed file. It is a file that was never finished. The most frequent cause is an interrupted write or transfer: a cluster job killed at the wall-time limit, a pipeline that crashed mid-stream, a network share that dropped, or a cp command that returned before the last block reached disk. Each of those events leaves the same fingerprint, a file that ends a few bytes short of the 28-byte BGZF end marker that signals a complete BAM.

The first thing to check is that marker. Run samtools quickcheck -v sample.bam. A clean exit with no output means htslib (the C library that samtools, Picard, and GATK all use to read BAM) found a valid header and a valid EOF block. A nonzero exit with a message like EOF marker is absent. The input is probably truncated sends you down the most fixable recovery path in the format. From there the work is mechanical: confirm the damage, isolate the unreadable region, salvage everything before and after it, and rewrite a clean file.

This guide covers the mechanics of the format, a triage sequence, a symptom table, the major failure modes from most common to rarest, concrete recovery commands, and prevention practices. Where a step can destroy data, the safety note is explicit.

What a BAM File Actually Is

BAM stands for Binary Alignment/Map. It is the compressed binary sibling of SAM (Sequence Alignment/Map), and both are defined by the SAM/BAM specification maintained in the hts-specs repository. A BAM file is not a single compressed stream. It is a sequence of independent gzip members, each one holding at most 64 KB of uncompressed data. That structure is called BGZF, or Blocked GNU Zip Format. Each member carries a small extra field that stores the compressed size of that block, which is what allows a reader to jump to an arbitrary virtual offset and decompress just that block. Random access depends entirely on this property.

The byte layout is fixed and worth memorizing because every recovery decision refers back to it:

1. The magic string BAM\1.
2. A 32-bit integer giving the length of the header text.
3. The header text itself, in SAM format, including @HD, @SQ, and @PG lines.
4. A 32-bit integer giving the number of reference sequences.
5. One record per reference sequence, with the name, length, and an MD5 checksum.
6. The alignment records, packed binary, spread across BGZF blocks.
7. The 28-byte BGZF EOF marker, an empty block with a known byte pattern.

The companion index (.bai for BAI, .csi for CSI) is a separate file. It maps reference intervals to virtual offsets, where a virtual offset is a 64-bit value combining the compressed block offset (upper 48 bits) and the offset within the decompressed block (lower 16 bits). If the BAM changes, the index is invalid. That single fact rules out a large class of apparent corruption.

Several other formats show up in the same conversations and are easy to confuse. FASTA stores sequences with no quality scores and no alignment, and it is used for reference genomes and protein sequences. FASTQ stores sequences plus per-base quality scores and is the raw output of most sequencers. SAM is the human-readable text form of BAM. CRAM is a reference-based compressed alignment format that stores differences from a reference genome and is substantially smaller than BAM for the same data. A file whose extension says .bam can be any of these if a pipeline mislabeled its output, and the first diagnostic step is often simply asking the file what it is.

Triage: The First Five Commands

Work through these in order before touching anything else. Each one is read-only.

1. Identify the format

htsfile sample.bam

A healthy file reports BAM version 1 compressed data. If it reports gzip compressed data, you may have a bgzipped SAM or a bgzipped VCF with the wrong name. If it reports ASCII text, you have a log file, an HTML error page, or a SAM file with a .bam extension. If htsfile is not installed, file sample.bam gives a coarser answer.

2. Check the EOF marker and header

samtools quickcheck -v sample.bam

The -v flag prints a message for every file that fails. Run it across a whole directory with samtools quickcheck -v *.bam and note which files fail. A nonzero exit status is the signal your pipeline should have been checking all along.

3. Read the header

samtools view -H sample.bam

If the header prints cleanly, the front of the file is intact and the reference list is recoverable. If this command fails, the damage starts inside the header itself, which is a different and harder problem.

4. Count records and scan for read errors

samtools view -c sample.bam

This decompresses every block in sequence. If it completes, the file is readable end to end and the problem is likely the index or a downstream tool. If it stops partway with an htslib error such as [E::bgzf_uncompress] Inflate operation failed, the message tells you the class of damage. Note how many records printed before the failure if you add -h and count, or run samtools view sample.bam | wc -l.

5. Validate against the specification

picard ValidateSamFile I=sample.bam MODE=SUMMARY

Picard's validator reports structural problems that samtools view tolerates: missing read groups, invalid mate flags, records out of coordinate order, and reference mismatches. GATK ships an equivalent tool as gatk ValidateSamFile -I sample.bam. Neither tool repairs anything. They tell you whether a readable file is also a valid file.

Troubleshooting Table

Causes, Mechanisms, and Fixes, From Most Common to Rarest

Missing EOF marker after an interrupted write

This is the single most common failure. Most writers, including samtools view, samtools sort, and Picard, close the output by writing the 28-byte empty BGZF block. If the process dies before that call, the file lacks the marker but may contain every alignment record. htslib treats the missing marker as an error because it cannot distinguish a clean finish from a truncation.

Check the last 28 bytes:

tail -c 28 sample.bam | od -An -tx1

The correct output is 1f 8b 08 04 00 00 00 00 00 ff 06 00 42 43 02 00 1b 00 03 00 00 00 00 00 00 00 00 00. If those bytes are present, the file ended cleanly and the problem lies elsewhere. If they are absent, the file is suspect, but not necessarily damaged.

Rewrite the file to restore the marker:

samtools view -b -o fixed.bam sample.bam

This reads every record and writes a fresh BGZF stream with a proper EOF block. If the read completes without an error, the file was complete and only the marker was missing. If the read stops early, you have genuine truncation and the rewrite contains whatever was salvageable. Always write to a new filename. Never redirect output over the input.

Truncated writes and partially flushed buffers

A truncation that removes the tail of the file loses more than the marker. The last BGZF block may be half-written, and every record in that block is gone. The size of the loss is bounded by the block size, at most 64 KB of uncompressed data, which for typical Illumina 150 bp reads is on the order of a few hundred alignments. In practice the loss is often smaller because the writer flushes blocks as they fill.

The recovery is the same rewrite shown above. samtools view -b -o recovered.bam broken.bam writes every record it can decompress and stops at the first block it cannot. The resulting file is valid and indexable. Records after the bad block are not recovered by this command alone, which is why the next section exists.

Mid-file BGZF block corruption

A block in the middle of the file can be damaged while later blocks remain perfectly intact. The classic causes are a bad sector on the storage device, a partial page write during a power loss, a filesystem that lost a block during an unclean shutdown, or a corrupted transfer over a network that did not checksum its payload. The htslib error is usually [E::bgzf_uncompress] Inflate operation failed: invalid distance too far back or invalid stored block lengths.

No standard tool silently skips a corrupt block. htslib reads sequentially and stops. Recovery therefore requires locating the boundary of the bad block and splitting the file into two readable regions.

Start by walking the block chain from the beginning. Each BGZF block starts with the four bytes 1f 8b 08 04, and the extra field contains a BC subfield whose value plus one gives the total block size. A short Python scan finds the first break:

import struct

MAGIC = b"\x1f\x8b\x08\x04"

def block_size(fh, offset):
    fh.seek(offset)
    head = fh.read(18)
    if len(head) < 18 or head[:4] != MAGIC:
        return None
    xlen = struct.unpack("<H", head[10:12])[0]
    extra = fh.read(xlen)
    i = 0
    while i + 4 <= len(extra):
        si1, si2, slen = struct.unpack("<ccH", extra[i:i+4])
        if si1 == b"B" and si2 == b"C":
            bsize = struct.unpack("<H", extra[i+4:i+6])[0]
            return bsize + 1
        i += 4 + slen
    return None

with open("sample.bam", "rb") as fh:
    fh.seek(0, 2)
    end = fh.tell()
    offset = 0
    while offset < end:
        size = block_size(fh, offset)
        if size is None:
            print("first invalid block at byte", offset)
            break
        offset += size
    else:
        print("block chain intact to EOF")

The script reports the byte offset of the first block whose header does not parse. Everything before that offset is the readable prefix. The suffix starts at the first valid block header after the damaged region. Find it by scanning forward for the magic bytes and testing each candidate with a small decompression attempt.

Create the prefix file and confirm it reads:

head -c OFFSET sample.bam > prefix.bam
samtools view -b -o prefix_clean.bam prefix.bam
samtools index prefix_clean.bam

Create the suffix by extracting from the restart offset and prepending a valid BAM header. The header is not present at the restart point, so build a header-only BAM from the prefix and concatenate:

samtools view -H prefix_clean.bam > header.sam
samtools view -b -o header_only.bam header.sam
tail -c +RESTART sample.bam > suffix.raw
cat header_only.bam suffix.raw > suffix.bam
samtools view -b -o suffix_clean.bam suffix.bam

The cat step works because BGZF readers tolerate a stream of concatenated blocks. The empty EOF block in the middle is ignored. The final samtools view -b normalizes the file and writes a new EOF marker at the true end. Extract SAM text from each half, concatenate, and rebuild once:

samtools view prefix_clean.bam > prefix.sam
samtools view suffix_clean.bam > suffix.sam
cat prefix.sam suffix.sam | samtools view -b -o merged.bam -
samtools sort -o merged_sorted.bam merged.bam
samtools index merged_sorted.bam

Expect a gap. The records inside the damaged block are gone, and a small number of records immediately after the restart point may be lost while the reader resynchronizes. Report the gap size and its approximate genomic location in your QC notes. If the lost region matters, re-align the corresponding reads from FASTQ.

Safety note: perform every step above on a copy of the file, never on the original, and never on a disk you suspect is failing. A rewrite that goes wrong on a failing drive can turn a recoverable file into an unrecoverable one.

Header damage

The header sits at the front and is compressed in the first BGZF block or two. Damage there is rarer than tail truncation because most writes touch the header once at the start. When it happens, usually from a partial first write or an in-place edit gone wrong, samtools view -H fails or prints a truncated reference list.

The fix is to transplant a healthy header from a sibling BAM produced by the same pipeline, or to reconstruct one from the reference FASTA used for alignment:

samtools faidx reference.fa
samtools view -H good_sample.bam > good_header.sam
samtools reheader good_header.sam broken.bam > reheadered.bam

samtools reheader rewrites only the header blocks and copies the alignment blocks unchanged, which makes it fast even on large files. The replacement header must list every reference sequence that appears in the alignment records, with matching names and lengths. A mismatch produces "chromosome not found" errors at index time. If you must rebuild the header from scratch, generate @SQ lines from the FASTA index and add a @PG line recording the recovery action and tool version.

Index corruption and mismatch

A corrupt index looks like a corrupt BAM. Region queries fail, samtools idxstats errors out, and IGV refuses to load the track. The BAM itself is usually fine. Index files are derived data and can always be rebuilt, so never spend time trying to repair one.

samtools index sample.bam

The BAM must be coordinate sorted. Confirm with the @HD line from samtools view -H, which should contain SO:coordinate. If it says SO:unsorted or SO:queryname, sort first:

samtools sort -@ 4 -o sorted.bam sample.bam
samtools index sorted.bam

A common trap is an index left over from a previous version of the BAM. After any rewrite, the old index is invalid even if the filename is unchanged. Delete it or overwrite it. Timestamps are a weak check because a copy operation can reset them, so the reliable check is to rebuild the index whenever the BAM changes.

For very large references with more than 512 megabases, BAI cannot address every position. Use CSI:

samtools index -c sample.bam

Format confusion

A surprising share of "corrupted BAM" reports turn out to be a file that was never a BAM. Pipeline wrappers that redirect stderr into the output path, cloud transfers that save an XML error document under the target key, and Snakemake or Nextflow jobs that fail before the first write all produce a file with a .bam extension and text inside. head -c 200 sample.bam answers the question in one second.

Related cases include bgzipped SAM saved as .bam, which samtools view -b -o fixed.bam mislabeled.bam converts correctly because htslib autodetects the input format, and CRAM saved with a .bam extension, which needs the reference:

samtools view -T reference.fa -b -o fixed.bam mislabeled.bam

CRAM decoding is reference-dependent by design. Without the exact reference used at encoding time, the file cannot be read, and no amount of repair will change that.

Concatenation and merge mistakes

Concatenating two BAM files with cat produces a file with two headers and two reference lists in the middle of the stream. htslib reads the first header and then encounters binary garbage where it expects alignment records. The error is usually an invalid record or an out-of-range block size.

Use samtools cat instead, which strips the redundant headers:

samtools cat -o combined.bam part1.bam part2.bam

samtools cat requires compatible headers. It does not sort, does not deduplicate, and does not fix read groups. For merging samples that should be analyzed together, use samtools merge, which adds read groups when asked:

samtools merge -o merged.bam -r sample1.bam sample2.bam
samtools index merged.bam

Both tools write a valid EOF marker, so the output passes quickcheck even if the inputs did not.

Filesystem and hardware failures

Bad sectors, failing SSDs, and network filesystems that lose writes produce mid-file corruption that no BAM tool can prevent. The first response is to stop reading from the source device. Every read of a failing drive risks a head crash or a firmware-level remap that makes later recovery harder.

Image the device first with GNU ddrescue, which is designed to skip bad regions and come back for them:

ddrescue -f -n /dev/sdb image.img image.map
ddrescue -d -r3 /dev/sdb image.img image.map

The first pass copies quickly and skips errors. The second pass retries with direct access and three retries per bad region. Then extract the BAM from the image, not from the device. If the BAM survives on the image, continue with the block-splicing procedure above.

Safety note: ddrescue writes to a new file and reads from a device. Confirm the source and destination arguments before you press enter. Reversing them overwrites the failing drive.

A Step-by-Step Recovery Protocol

This sequence assumes a single BAM that fails quickcheck and is worth recovering. Adapt the filenames.

1. Copy or image the file. Never operate on the only copy.
2. Run htsfile to confirm it is a BAM.
3. Run samtools quickcheck -v and record the exact error.
4. Run tail -c 28 file.bam | od -An -tx1 and compare with the standard EOF block.
5. Run samtools view -H to test the header.
6. Run samtools view -c and note whether it completes and where it stops.
7. If the header reads and the count completes, the file is intact. Rebuild the index and move on.
8. If the header reads and the count fails mid-file, run the Python block scan to find the first bad offset.
9. Extract the prefix with head -c, rewrite it with samtools view -b, and index it.
10. Find the first valid block after the bad region, extract the suffix, prepend a header-only BAM, and rewrite it.
11. Concatenate the SAM text from both halves, rebuild, sort, and index.
12. Run samtools flagstat on the result and compare with the expected yield from the sequencing run.
13. Run Picard or GATK validation on the final file.
14. Record the recovered and lost record counts in your pipeline log.

The whole sequence takes minutes for files up to tens of gigabytes because the prefix rewrite is sequential and the suffix work touches only the tail.

Salvaging Reads and Converting Between Formats

Recovering alignments is often not the end goal. Many workflows only need the reads back so they can be re-aligned with a corrected reference or a newer aligner. BAM stores the sequence and quality string for every record, including unmapped reads if the aligner kept them, so FASTQ can be regenerated.

samtools collate -o collated.bam recovered.bam
samtools fastq -1 R1.fastq -2 R2.fastq -0 other.fastq -s single.fastq collated.bam

samtools collate groups reads by name without loading the whole file into memory, which keeps paired mates together. samtools fastq then splits them into files by mate status: read 1, read 2, records with no mate or an unclear mate, and singleton reads. If the aligner discarded unmapped reads (a common setting with -F 4 downstream, or an aligner that only stores mapped reads), those reads are not in the BAM and cannot be recovered from it. They must come from the original FASTQ.

The distinction between FASTQ and FASTA matters here. FASTQ carries four lines per read: an identifier, the sequence, a separator, and a quality string encoded as ASCII characters offset by 33. FASTA carries two lines per record, an identifier and the sequence, with no quality scores. Converting BAM to FASTA is possible with samtools fasta, and the conversion silently discards quality information, which means the reads can never be re-aligned with full base-level confidence afterward. Convert to FASTQ unless you have a specific reason to do otherwise, and keep the FASTQ output as the canonical recoverable form.

Compression is worth a note. Plain FASTQ for a human genome at 30x is hundreds of gigabytes. samtools fastq can write to standard output, so pipe it into a compressor:

samtools fastq -1 >(gzip > R1.fastq.gz) -2 >(gzip > R2.fastq.gz) collated.bam

Process substitution requires bash. In other shells, write uncompressed and compress afterward.

Prevention

Recovery is expensive and incomplete. Avoiding corruption is cheaper.

• Check every BAM as it is written. Add samtools quickcheck -v as a pipeline step and fail the job on a nonzero exit. One line prevents most of the cases in this article.
• Write to a temporary name and move on success. samtools sort -o final.tmp.bam followed by mv final.tmp.bam final.bam guarantees that a downstream consumer never sees a half-written file. mv within a filesystem is atomic.
• Use set -o pipefail in every shell script. Without it, a failed upstream command in a pipeline is masked by the successful exit of the downstream command, and the pipeline reports success while writing a truncated file.
• Checksum at rest and in transit. Record sha256sum at write time, verify after every transfer, and use rsync -c when integrity matters more than speed. Most cloud object stores expose an ETag or checksum that can be compared.
• Keep the index with the BAM. A stale index is the second most common cause of "corrupted BAM" reports. Rebuild the index in the same job that writes the BAM.
• Use redundant storage for primary data. RAID is not a backup. A copy on a second filesystem or in object storage with versioning protects against the failure modes that no tool can repair.
• Prefer CRAM when storage is the constraint. CRAM with a recorded reference is substantially smaller than BAM and its reference-based encoding makes some forms of damage easier to detect, but it adds a dependency on the exact reference. Store the reference checksum alongside the CRAM.
• Keep the original FASTQ. The BAM is derived data. As long as the FASTQ exists, a damaged BAM costs compute time, not data.

Common Mistakes and Limitations

• Overwriting the input. samtools view -b broken.bam > broken.bam truncates the input before reading it. Always use -o with a new name.
• Assuming quickcheck proves integrity. It validates the header and the EOF marker only. A file can pass and still have a corrupt middle block. Run samtools view -c periodically on critical datasets.
• Rebuilding an index and calling it recovery. A new index on a truncated BAM hides the truncation from region queries while the missing records stay missing.
• Treating a header-only BAM as success. If the prefix rewrite yields zero alignment records, you have recovered a header, not data.
• Forgetting read groups. A recovered BAM that passes validation may still lack @RG lines, which breaks GATK's base quality score recalibration and most joint-calling workflows. Check samtools view -H and add read groups with Picard AddOrReplaceReadGroups if needed.
• Ignoring sort order after splicing. The concatenated result is not guaranteed to be coordinate sorted even if both halves were. Sort before indexing.
• Duplicating records at the splice boundary. A few records near the restart point can appear in both halves. Run samtools markdup after samtools fixmate if duplicate counts matter, or deduplicate by read name.
• Trusting timestamps to detect a stale index. Copy operations reset them. Rebuild the index whenever the BAM changes.
• Reading repeatedly from a failing disk. Each read risks further damage. Image first.
• Expecting a full recovery. Mid-file corruption always loses the records inside the damaged block. The honest output of a recovery effort is a recovered count, a lost count, and a note on which genomic region was affected.

Safety note: the recovery commands in this article write new files. Disk space is the limiting resource. Check free space with df -h before extracting a multi-gigabyte prefix and suffix, and delete intermediate files as you confirm each step.

Frequently Asked Questions

Can I fix a corrupted BAM file without losing any reads?

Sometimes. A file that is missing only the EOF marker, with all records intact, is fully recoverable with a single rewrite. A file truncated mid-record always loses the last block. A file with a corrupt middle block always loses the records inside that block.

Does a missing EOF marker mean my BAM is corrupted?

No. It means the writer did not finish cleanly. The records may all be present. Check the last 28 bytes, then run a full record count to find out whether the data itself is complete.

How do I recover FASTQ data from a BAM file?

Run samtools collate to group reads by name, then samtools fastq with -1, -2, -0, and -s to split the output by mate status. Only reads stored in the BAM can be recovered. If the aligner discarded unmapped reads, those must come from the original FASTQ.

What is the difference between FASTQ and FASTA?

FASTQ stores a sequence plus per-base quality scores, four lines per read. FASTA stores a sequence only, two lines per record. FASTA is used for references and protein sequences. FASTQ is the raw read format and the one you want when recovering data for re-alignment.

Why does samtools say "EOF marker is absent"?

The file does not end with the 28-byte empty BGZF block that marks a completed BAM. The usual cause is a process that died before its final flush, or a transfer that stopped early. Rewrite the file with samtools view -b -o fixed.bam broken.bam to add the marker.

Should I repair a BAI index or rebuild it?

Rebuild it. Indexes are derived data with no independent value. Delete the old file and run samtools index on a coordinate-sorted BAM. Use samtools index -c for references larger than 512 megabases.

Can I merge recovered BAM fragments into one file?

Yes. Extract SAM text from each fragment, concatenate the text, and rebuild with samtools view -b. Sort and index the result. Use samtools cat for intact files with compatible headers, or samtools merge when you need read groups added.

What is the first command to run on a suspect BAM?

samtools quickcheck -v sample.bam. It tests the header and the EOF marker in one pass and returns a nonzero exit status on failure, which makes it easy to script.