BAM File Metadata and Validation

By Dr. Zubair Khalid, DVM, MS, PhD · 2026-11-10

Most BAM metadata validation failures trace back to one thing: the header does not describe the data that follows it. Either the read group block is missing, the sort order does not match what the file actually contains, or the reference dictionary names contigs the aligner never used. The single most common cause is a missing or incomplete @RG read group line, because many aligners emit none unless you explicitly ask. The first thing to check is not the alignments but the file itself and its header, with two commands:

samtools quickcheck -v sample.bam
samtools view -H sample.bam

If quickcheck is silent, the file has a valid header and a complete BGZF end-of-file block. If it prints a filename, the file is truncated, unreadable, or not BAM at all. The header dump then shows whether the metadata downstream tools depend on is present, consistent, and internally coherent. Everything else in this guide builds from those two results.

BAM stands for Binary Alignment/Map, the compressed binary encoding of the SAM (Sequence Alignment/Map) text format. The SAM specification defines both the header and the record structure. Validation means checking that a BAM conforms to that specification, that its header agrees with its records, and that its header agrees with the reference genome it claims to be aligned against. This article covers the metadata layer in depth, walks through causes from most to least frequent, gives a diagnostic sequence you can run in order, and closes with prevention practices that keep validation failures out of production pipelines.

What BAM Metadata Actually Contains

The BAM header is plain ASCII text stored uncompressed at the very start of the file, before any alignment record. It is short, usually a few kilobytes even for a whole human genome, and it is where every metadata check begins. Header lines begin with @ and are grouped by record type.

The Five Header Record Types

@HD appears exactly once and must be the first line. It carries VN, the SAM format version, and SO, the sort order. Valid SO values are unknown, unsorted, queryname, and coordinate. A SS sub-sort field can add detail such as coordinate:minhash. The SO value is a promise about record ordering, and index construction depends on that promise being true.

@SQ appears once per reference sequence and defines the reference dictionary. SN is the sequence name and LN is its length in bases. These two fields are mandatory. M5 is the MD5 checksum of the uppercase reference sequence with whitespace removed, UR is a URI pointing at the reference file, AS names the assembly, and SP names the species. A dictionary with M5 values lets any downstream tool prove that the BAM and the FASTA in hand are the same molecules, not just the same names.

@RG is the read group. ID is mandatory and must be unique within the file. SM is the sample name, LB is the library, PL is the sequencing platform, PU is the platform unit (typically flowcell and lane), CN is the sequencing center, and DT is the run date in ISO 8601 form. Every alignment record carries an RG:Z: tag whose value must match one of these ID values. This is the field set that variant callers use to decide which reads belong to which sample.

@PG records the program that touched the file. ID is unique, PN is the program name, VN is the version, CL is the full command line, and PP points to the ID of the previous program in the chain. A well-formed @PG chain is a complete processing history from raw reads to final BAM.

@CO holds free-text comments and has no structure.

Where the Metadata Comes From

Metadata is not written by the sequencer. It is written by the aligner, and then appended to by every sorting, duplicate-marking, and filtering step. If the aligner is told nothing about samples, the BAM will say nothing about samples. That single design fact explains most of the failures described below. The same principle shows up across data-intensive biology: when sample properties are not bound to data files in a standardized way, reuse and reproduction suffer, which is the problem that structured metadata mapping is designed to solve in proteomics repositories [1]. Alignment files are no different.

Causes of BAM Metadata Validation Failures, From Most to Least Common

1. Missing or Incomplete Read Groups

This is the most frequent error reported by Picard and GATK validators. Burrows-Wheeler Aligner (bwa) and several other aligners write no @RG line unless you pass a read group string. The resulting BAM is structurally valid but functionally anonymous. Picard reports MISSING_READ_GROUP as a warning. GATK's HaplotypeCaller treats the absence of a sample name as fatal, because it cannot label the output VCF.

The related error is READ_GROUP_NOT_FOUND, which fires when records carry an RG:Z: tag pointing to an ID that does not exist in the header. This happens after merging BAMs, after subsetting, or after a header was replaced without updating the tags.

A third variant is a read group that exists but is under-specified. A @RG with only ID and SM will pass some validators and fail others. Population and clinical workflows generally want ID, SM, LB, PL, and PU populated for every read group.

2. Sort Order and Index Mismatch

samtools index requires coordinate-sorted input. If the records are not in coordinate order, indexing fails with an unsorted positions error. The trap is that @HD SO:coordinate can be a lie if a file was concatenated, filtered, or rewritten by a tool that did not re-sort. Validators that trust the header will pass the file, and the indexer will fail later.

Queryname sort is a special case. samtools sort -n produces a file with SO:queryname, which is correct and necessary for fixmate and some duplicate-marking paths, but it cannot be indexed and cannot be used for interval queries. A BAM left in name-sorted order after duplicate marking is one of the most common silent causes of downstream failure.

3. Reference Dictionary Mismatch

Every @SQ line is a claim about the reference. If those claims do not match the FASTA used downstream, tools either refuse to run or, worse, run and produce wrong results. The classic triggers are contig naming (chr1 versus 1), different assembly releases (GRCh37 versus GRCh38), a reference that includes alt or decoy contigs when the BAM was aligned without them, and a BAM aligned to a transcriptome but analyzed against a genome.

When M5 checksums are present on both sides, the mismatch is provable rather than suspected. When they are absent, you are reduced to comparing names and lengths. Formal consensus guidance for clinical genomics recommends adopting hg38 as the reference build and using standardized file formats with strict version control, precisely because reference ambiguity propagates into every downstream result [2].

4. Truncated or Corrupted Files

A BAM file ends with a specific 28-byte BGZF end-of-file marker. If a transfer is interrupted, a disk fills, or a process is killed mid-write, the marker is absent and the file is truncated. samtools quickcheck detects this immediately. Files pulled from cloud storage or transferred over unreliable links are the usual culprits, and the failure can be invisible to ls because the file size looks plausible.

Corruption inside the compressed stream is rarer but possible. It usually surfaces as a CRC or inflate error when a specific region is read, not at open time.

5. Conflicting Sample and Library Labels After Merging

Merging lanes is routine and merging is where sample labels drift. Two failure patterns dominate. First, the same SM value is used for two genuinely different samples, which silently pools them during joint genotyping. Second, the same read group ID is reused across merged inputs with different SM, LB, or PU content, so the header has duplicate ID values and the tags become ambiguous.

Multi-library samples create a legitimate variant of this problem. One sample sequenced from two libraries should have two @RG lines sharing one SM and differing in LB. If both libraries are collapsed into one read group, duplicate marking and library-aware error models lose the information they need.

6. Missing or Broken Provenance Records

A @PG chain that stops abruptly, or that contains no CL fields, makes a BAM unauditable. You cannot tell which aligner version produced it, which reference was used, or what filtering was applied. Public health and clinical programs treat this as a real defect because reproducible, validated, and auditable results are an explicit requirement of production bioinformatics, and standardized outputs plus version-controlled code are how that requirement is met [3]. Containerizing the toolchain with pinned versions closes the loop, since the recorded version then maps to a specific, reproducible image [4].

7. Vocabulary and Format Violations

These are small, mechanical, and easy to fix once found. PL must be one of the values defined by the SAM specification, such as ILLUMINA, IONTORRENT, PACBIO, OXFORDNANOPORE, DNBSEQ, LS454, SOLID, CAPILLARY, HELICOS, ELEMENT, ULTIMA, or SINGULAR. Writing Illumina or illumina instead of ILLUMINA will trip strict validators. DT must be ISO 8601. Header text must be printable ASCII. Duplicate @SQ names or duplicate @RG IDs within one header are invalid.

8. Record-Level Errors

These are not header problems but they surface in the same validation run. SEQ and QUAL must have equal length, or QUAL must be *. MAPQ must be an integer from 0 to 255. CIGAR operations must not extend beyond the declared reference length. Flag bits must be mutually consistent, so a read cannot be simultaneously marked unmapped and have a mapped mate position that contradicts it. bin values must match the record position, which is what makes BAI lookup work. These errors usually point at a broken aligner run, a hand-edited SAM, or a tool that wrote records outside the specification.

Troubleshooting Table

How to Fix BAM File Metadata Validation: A Diagnostic Sequence

Run these steps in order. Each one narrows the space before you commit to a fix.

Step 1. Check file integrity. samtools quickcheck -v sample.bam verifies that the file is BGZF, has a header, and ends with the correct marker. Pair it with md5sum sample.bam and compare against the checksum recorded when the file was created. A truncated file cannot be repaired by any header edit, so resolve this before anything else.

Step 2. Dump the header. samtools view -H sample.bam shows every header line. Confirm @HD exists with a VN value and a declared SO. Count the @SQ, @RG, and @PG lines. A file with zero @RG lines needs Step 4. A file with zero @PG lines has no provenance.

Step 3. Verify the sort order against reality. Read the first few thousand records with samtools view sample.bam | head -n 1000 and check whether the leftmost coordinate increases. If SO:coordinate is declared but the coordinates wander, the header is false and the file needs re-sorting. samtools flagstat sample.bam gives a quick independent summary of mapped, properly paired, duplicate, and unmapped counts that helps confirm the file is what you think it is.

Step 4. Audit the read groups from both directions. List the declared groups with samtools view -H sample.bam | grep '^@RG'. Then list the tags actually present on records with samtools view sample.bam | grep -o 'RG:Z:[^[:space:]]*' | sort | uniq -c. The two lists must agree. A tag value with no matching ID is an error. An ID used by no reads is usually harmless but worth noting.

Step 5. Compare the reference dictionary. Generate a dictionary from the FASTA you intend to use: samtools faidx ref.fa then samtools dict -a GRCh38 -s Homo_sapiens ref.fa > ref.dict. Diff its @SQ lines against the BAM header as shown in the table. Names, order, and lengths should all match. If M5 checksums are present on both sides, compare them directly, since a length match with a checksum mismatch means the sequences differ.

Step 6. Check the index. samtools idxstats sample.bam requires a valid index and prints one line per contig with mapped and unmapped counts. It is a fast way to confirm that the index exists, covers the same contigs as the header, and that reads landed where you expect. A contig with a header length of hundreds of megabases and zero mapped reads usually means a naming mismatch rather than a biological surprise.

Step 7. Run a formal validator. picard ValidateSamFile I=sample.bam MODE=VERBOSE R=ref.fa reports errors and warnings with precise record coordinates where relevant. GATK exposes the same machinery through gatk ValidateSamFile -I sample.bam, REFERENCE ref.fa -MODE VERBOSE. Read the error categories rather than the count. Ten instances of one systematic error is one bug, not ten.

Step 8. Sanity-check the alignments. samtools stats sample.bam | grep '^SN' reports error rate, insert size distribution, and read length statistics. A misplaced sample, a swapped lane, or a wrong reference often shows up as an implausible insert size or error rate before any validator complains.

Fixing the Header Without Realigning

Header-only defects can be repaired without touching alignment records. Two rules keep this safe.

Use samtools addreplacerg when read groups are missing or wrong:

samtools addreplacerg \
  -r 'ID:flowcell1.lane1\tSM:NA12878\tLB:lib1\tPL:ILLUMINA\tPU:flowcell1.1' \
  -m overwrite_all \
  -o with_rg.bam input.bam

The \t sequences must be literal tab escapes, which samtools interprets. Picard does the same job with typed arguments:

picard AddOrReplaceReadGroups \
  I=input.bam O=with_rg.bam \
  RGID=flowcell1.lane1 RGSM=NA12878 RGLB=lib1 \
  RGPL=ILLUMINA RGPU=flowcell1.1

Use samtools reheader for structural header edits, and always write to a new file:

samtools view -H input.bam > header.sam
# edit header.sam in a text editor
samtools reheader header.sam input.bam > reheadered.bam
samtools index reheadered.bam

Data safety note: reheader rewrites the file. Never do it in place on your only copy, and never run it on a file whose checksum you have not recorded. Also note that @RG fields often contain subject or patient identifiers. Treat header text as protected health information when the sample is human, and scrub or pseudonymize SM before sharing BAMs outside your institution.

samtools reheader also adds a @PG line by default when writing. That is desirable. It records that a reheader happened, with the command line, which is exactly the audit trail you want.

BAM File Metadata Validation Solutions by Workflow Stage

At Alignment Time

Write the read group at the source. For bwa mem, the read group string is passed with -R and the escapes are interpreted by bwa:

bwa mem -t 8 \
  -R '@RG\tID:flowcell1.lane1\tSM:NA12878\tLB:lib1\tPL:ILLUMINA\tPU:flowcell1.1' \
  ref.fa R1.fastq.gz R2.fastq.gz \
  | samtools sort -@ 4 -o sample.sorted.bam -
samtools index sample.sorted.bam

This is the cheapest possible intervention. Every downstream problem listed in this article gets easier when the read group exists from the first BAM onward. Do not defer it to "later cleanup" unless a LIMS system will supply the values programmatically.

After Sorting and Duplicate Marking

Duplicate marking requires name sorting and mate fixing before coordinate sorting. The correct order for samtools markdup is:

samtools sort -n -o namesort.bam aligned.bam
samtools fixmate -m namesort.bam fixmate.bam
samtools sort -o positionsort.bam fixmate.bam
samtools markdup positionsort.bam markdup.bam
samtools index markdup.bam

Two metadata points matter here. samtools fixmate -m adds the mate score tag that markdup needs, and skipping it produces flags that validators later flag as inconsistent. Then confirm that @HD SO:coordinate survived the final sort and that the index was rebuilt. An index built before duplicate marking is stale even though it still opens.

Before Variant Calling

Before a BAM enters a caller, confirm four things: at least one @RG exists, exactly one SM value is present per sample, PL is a valid platform, and the @SQ dictionary matches the reference passed to the caller. GATK will fail loudly on the first and third, and quietly on the second and fourth only insofar as the output sample name or the variant coordinates will be wrong.

Sample identity is the failure mode that matters most. If two BAMs for the same individual carry different SM values, joint genotyping treats them as two individuals. If two individuals share an SM value, they are pooled as one. Neither error produces an exception. The reproducible answer is a sample sheet that maps every library and platform unit to a single canonical sample identifier, then generating read group strings from that sheet rather than typing them by hand.

Before Archiving or Submission

At deposit time, verify that the @PG chain is intact, that M5 and UR are present on @SQ lines, that the index is current, and that the file passes a full ValidateSamFile run without errors. Record the file checksum alongside the file. Auditable results depend on the chain from sample sheet to read group to final BAM being unbroken, which is the same requirement that drives validated, standardized, and version-controlled output generation in production public health workflows [3].

Pipelines should encode these checks as stages, not as ad hoc commands. Assembly quality pipelines in wide use perform file-format validation as an explicit step and emit the results into a structured report, so a failing input stops the run with a readable diagnostic rather than surface later [5].

Metadata Standards and Reproducibility

The reason validation rules feel pedantic is that headers are the only machine-readable statement of what a BAM contains. When those statements are inconsistent, results stop being comparable across runs, instruments, and laboratories. The reproducibility problem in bioinformatics is well documented as a consequence of inconsistent standardization, and tooling that pins environments and versions is one response to it [4]. The same logic applies at the file level: a BAM with pinned reference M5 values, complete read groups, and a full @PG chain can be compared against another BAM with confidence.

Formal guidance for clinical genomics goes further. Consensus recommendations from production clinical bioinformatics units include adopting hg38 as the reference build, standardizing file formats, applying strict version control, and running at quality standards comparable to recognized laboratory accreditation frameworks [2]. Each of those recommendations lands directly on BAM metadata: the build choice is encoded in @SQ, the file format choice determines which validators apply, and version control determines whether the recorded @PG entries mean anything.

Standardized metadata also improves reuse after deposit. In proteomics, the recognition that inconsistent repository metadata blocks reuse led to tooling that exports sample properties in a standard structured format mapping each sample to its data files [1]. Alignment data has the same need, met less formally through @RG. Treat the read group block as the equivalent sample-to-file map for sequencing data.

Bioinformatics File Formats: Where BAM Sits

Understanding the format family makes the validation rules easier to remember, because each format carries a different metadata load.

FASTQ vs FASTA

FASTA is the simplest format: a header line beginning with > followed by a sequence, with no per-base quality. It is used for reference genomes, transcriptomes, and assembled contigs. FASTA carries no quality information by design.

FASTQ carries both sequence and quality. Each record has four lines: an @ header, the sequence, a + separator line, and a quality string of exactly the same length as the sequence. Each quality character encodes a Phred score offset by 33 in the standard encoding, so ASCII I (decimal 73) represents Q40. The length equality between sequence and quality is the FASTQ analog of the SEQ/QUAL rule enforced inside BAM validation.

FASTQ is raw read data. FASTA is reference or assembly data. Neither is an alignment format, and neither carries read group or reference dictionary metadata in any standardized form. The header line of a FASTQ may hold instrument and run information, but nothing enforces it.

SAM, BAM, and CRAM

SAM is the plain-text alignment format defined by the specification, with the same header block described earlier followed by tab-delimited alignment records. BAM is the BGZF-compressed binary encoding of SAM, with the header stored as plain text at the start and the records stored in a compact binary structure. BAM is the working format for most short-read pipelines because it is compact, indexable, and universally supported.

CRAM is a reference-based compression format. It stores reads as differences from a reference, so decoding requires the same reference FASTA and produces much smaller files. CRAM carries the same logical metadata as BAM, including @SQ, @RG, and @PG, but the reference dependency adds a new failure mode: a CRAM with a mismatched reference dictionary will not decode at all.

FASTA and structured annotation formats interoperate through explicit metadata fields, and standardization efforts for annotation data show the value of a documented format that binds method details to sequence files so results can be compared across studies [6]. The same habit, binding method and sample details to the data file itself, is what a complete BAM header achieves.

Index Files

A BAI index is a separate file that maps genomic intervals to file offsets so tools can seek without reading the whole BAM. It requires coordinate-sorted input. The BAI format has a hard limit: it cannot index references longer than 2^29 minus 1 bases, roughly 512 megabases. Most human and model organism contigs fit. Many plant and amphibian contigs do not, and those references require a CSI index, which supports much longer sequences.

The index is not metadata, but index failures are the most common symptom of metadata failures. A BAM whose header claims coordinate sort but whose records are unsorted will produce an unsorted positions error at index time, which is really a header truthfulness problem.

Common Mistakes and Limitations

Trusting the header over the records. Validators that read only @HD will pass a file whose records contradict it. Always spot-check record order directly.

Using reheader to force a dictionary match. If a BAM was aligned to GRCh37 and you reheader it with GRCh38 @SQ lines, the coordinates now point at the wrong bases. Validators may pass and the variants will be wrong. Reheader only when the underlying sequences are provably identical, which the M5 checksums can establish.

Assuming MISSING_READ_GROUP is cosmetic. Picard classifies it as a warning. GATK treats missing sample information as fatal. The severity depends on the consumer, not on the file.

Reusing read group IDs across merged inputs. Duplicate ID values with different content make tags ambiguous. Generate unique IDs from flowcell, lane, and library.

Renaming a VCF sample to fix a BAM metadata problem. The VCF header will now disagree with the BAM it came from, and any reanalysis will reproduce the original error. Fix SM in the BAM before calling.

Forgetting to rebuild the index after any record-changing step. A stale .bai opens without complaint and returns wrong intervals.

Skipping fixmate before markdup. Duplicate marking produces flags that look inconsistent to validators, and the resulting duplicates are unreliable.

Ignoring the EOF marker after a download. Cloud transfers fail silently more often than local copies. quickcheck costs a fraction of a second.

Treating a passing validator run as proof of correctness. ValidateSamFile checks structure, not biology. A BAM can pass every structural check and still be the wrong sample, the wrong reference build, or contaminated. Metadata validation is necessary and not sufficient. For any individual case where sample identity or clinical interpretation is in question, consult the responsible bioinformatician or laboratory director rather than relying on a validator's exit code.

Frequently Asked Questions

What does samtools quickcheck actually check?

It checks that the file is BGZF-compressed, that it begins with a readable header, and that it ends with the correct end-of-file block. It does not examine alignment records, so it detects truncation and gross format errors but says nothing about metadata content.

Why does GATK report no read groups when the header looks fine?

The header probably lacks an @RG line entirely, which is common for bwa output produced without the -R option. Dump the header and search for lines beginning with @RG. If none exist, add them with samtools addreplacerg or re-align with a read group string.

Can I fix BAM metadata without realigning?

Yes for header-only defects such as missing read groups, wrong platform values, or inconsistent sample names. No for reference dictionary mismatches that reflect a genuinely different assembly, because the alignment coordinates themselves would be wrong.

Why does samtools index fail with an unsorted positions error?

The records are not in coordinate order, regardless of what the header says. Sort with samtools sort, then run samtools index on the sorted output. If the file was name-sorted for duplicate marking, that step must be followed by a coordinate sort.

How do I confirm a BAM matches a specific reference FASTA?

Compare the @SQ lines from the BAM header against a dictionary generated with samtools dict from the FASTA. Names, order, and lengths should match, and where present the M5 MD5 checksums should be identical.

What is the difference between a BAI and a CSI index?

Both are coordinate indexes. BAI cannot handle reference sequences longer than roughly 512 megabases, while CSI supports much longer sequences. Use CSI for large plant or amphibian genomes and BAI elsewhere unless your tools require otherwise.

Does a BAM file store FASTQ quality scores?

Yes, but not as ASCII text. BAM stores quality as raw Phred bytes, one per base, and the count must equal the sequence length. The ASCII offset used in FASTQ is removed at conversion time and reapplied only when converting back to SAM text.

Why is my VCF sample name wrong after joint genotyping?

The input BAMs had inconsistent SM values in their @RG lines. Harmonize sample names across all input BAMs before calling, then regenerate the VCF rather than renaming samples in the output.

Sources

1. Facilitating analysis and dissemination of proteomics data through metadata integration in MaxQuant.
2. Recommendations for bioinformatics in clinical practice.
3. Accelerating bioinformatics implementation in public health.
4. CREDO: a friendly Customizable, REproducible, DOcker file generator for bioinformatics applications.
5. AssemblyQC: a Nextflow pipeline for reproducible reporting of assembly quality.
6. DeNoFo: a file format and toolkit for standardized, comparable de novo gene annotation.