# SAM File Validation: How to Check By the end of this guide you will be able to take any SAM, BAM, or CRAM file and decide, with evidence, whether it is structurally valid, internally consistent, and safe to hand to a downstream tool such as a variant caller, a quantification pipeline, or a long-term archive. You will know which command gives a two-second pass or fail, which command gives a full audit, and how to tell the difference between a file that merely parses and a file that is actually correct. Have these on hand before you start: - samtools 1.16 or newer (1.20 is the current release line) - Picard 3.x, which ships the `ValidateSamFile` tool - Python 3.9 or newer with `pysam` installed - The reference FASTA that produced the alignments, plus its `.fai` index - The SAM/BAM specification from hts-specs, which is the normative document for every field discussed here Nothing in this workflow writes to your input file. Every command below is read-only, so you can run the full sequence on production data without risk. ## What a SAM File Is and Where It Sits Among Bioinformatics File Formats SAM stands for Sequence Alignment/Map. It is a tab-delimited plain-text format that stores reads after they have been aligned to a reference. Each line is one alignment record, and each record has eleven mandatory fields followed by any number of optional tags written as `TAG:TYPE:VALUE`, such as `NM:i:3` or `RG:Z:grp1`. Lines beginning with `@` are header lines. They are not alignment records and they must come first. The five header record types are `@HD` (file-level metadata such as version and sort order), `@SQ` (reference sequence dictionary, one line per contig), `@RG` (read group), `@PG` (program record, one per program that touched the file), and `@CO` (free-text comment). ### The eleven mandatory fields | # | Field | Meaning | Example | |--|--|--|--| | 1 | QNAME | read name | `A00123:8:H3V7LDSX2:1:1101:1000:1000` | | 2 | FLAG | bitwise status code | `99` | | 3 | RNAME | reference contig name | `chr1` | | 4 | POS | 1-based leftmost mapping position | `100` | | 5 | MAPQ | mapping quality, Phred-scaled | `60` | | 6 | CIGAR | alignment operations | `8M2I4M1D3M` | | 7 | RNEXT | mate reference name, `=` for same contig | `=` | | 8 | PNEXT | mate leftmost position | `300` | | 9 | TLEN | observed template length | `216` | | 10 | SEQ | read sequence | `TTAGATAAAGGATACTG` | | 11 | QUAL | per-base quality, Phred+33 ASCII | `IIIIIIIIIIIIIIIII` | Every one of these fields is checkable. Most validation failures come down to a mismatch between two of them, for example a CIGAR that implies 17 query bases while SEQ contains 15. ### FASTQ vs FASTA vs SAM These three formats are often confused because they all carry [nucleotide sequence](/knowledge/molecular-biology/nucleotide-sequence). They answer different questions. | Format | Carries | Quality scores | Alignment to a reference | Relative size | |--|--|--|--|--| | FASTA | sequence only | no | no | smallest | | FASTQ | sequence plus one quality character per base | yes | no | roughly 2x the sequence | | SAM | aligned reads, text | yes | yes | largest, often 5 to 10x FASTQ | | BAM | binary, BGZF-compressed SAM | yes | yes | roughly one third of SAM | | CRAM | reference-based compression | yes | yes | smallest aligned format | FASTA is a container for reference sequence, assembly contigs, or protein. FASTQ is the raw read format that comes off an Illumina or Nanopore run, with four lines per record: an `@` header, the sequence, a `+` separator, and a quality string the same length as the sequence. SAM is what you get after those FASTQ reads have been mapped. A SAM record with FLAG bit 0x4 set (unmapped) still carries the original sequence and quality, so a SAM file can technically hold the same information as a FASTQ file, but it is a far more expensive way to store it. The practical consequence for validation is that a FASTQ file has only two length invariants (sequence length equals quality length, and each record has four lines), while a SAM file has a dozen interacting invariants across header, CIGAR, SEQ, QUAL, FLAG, and mate coordinates. That is why SAM validation needs tooling rather than a quick eyeball. ## Why SAM File Validation Matters A corrupt or inconsistent alignment file rarely fails at the point of corruption. It fails hours later, inside a variant caller that crashes on record 4,000,001, or worse, inside an analysis that completes and produces a plausible but wrong result. Catching the problem at ingestion is far cheaper than catching it at interpretation. PipeVal was built explicitly for this problem. It is a light-weight, extensible validation tool designed to be inserted into existing methods and pipelines so that inputs and outputs are automatically verified, reducing wasted compute time attributed to file corruption or invalid file paths [1]. The design principle generalizes: validate at the boundary, not in the middle of a run. Validation also protects reproducibility. A formal reliability validation procedure for file interpretation requires documenting the forensic process, the tools used, and the method, so that the result can be reproduced independently [2]. The same standard applies to alignment files in research pipelines. If you cannot state which version of samtools parsed your BAM and what checks passed, you cannot defend the file later. Standardization efforts in other domains make the same point. A unified validation framework for EEG-based seizure detection algorithms had to specify datasets, file formats, input content, annotation formats, and reporting metrics before results from different groups could be compared at all [3]. Alignment files sit at the same kind of boundary. Without a defined validation step, two labs can both claim to have "valid BAMs" while holding files with different header completeness, different sort orders, and different tag conventions. ## Two Levels of Validation: Syntax and Semantics It helps to separate two questions that are often blurred together. **Syntactic validation** asks whether the file conforms to the specification. Are there eleven or more tab-separated fields per line? Does the header contain at least one `@SQ` line? Is the CIGAR string composed of valid operators? Does SEQ length equal QUAL length? Does the CIGAR query length equal SEQ length? Is the FLAG an integer in range? **Semantic validation** asks whether the file is internally consistent and biologically plausible. Does the `@SQ` length match the actual reference? Is the file really coordinate-sorted when `@HD SO:coordinate` claims it is? Are paired reads actually adjacent-compatible? Do reads with the 0x2 proper-pair bit set have mates that point back at them? Do the `NM` and `MD` tags agree with the CIGAR and the reference? Most tools do the first well and the second partially. A file that passes `samtools view` has passed a parse. That is not the same as passing validation. Plan for both levels. ## What You Need Before You Start Install the core tools once. Conda is the least painful route on Linux and macOS. ```bash conda create -n samval -c bioconda -c conda-forge \ samtools=1.20 picard=3.3.0 pysam=0.22.0 htslib=1.20 conda activate samval samtools --version ``` The version output should look like this: ``` samtools 1.20 Using htslib 1.20 Copyright (C) 2024 [Genome Research](/blog/guides/genome-research) Ltd. ``` If you prefer containers, the BioContainers images are drop-in replacements and avoid dependency drift between machines. ```bash docker run --rm -v "$PWD":/data quay.io/biocontainers/samtools:1.20--h50ea8bc_1 \ samtools --version ``` Confirm the reference index exists. If `ref.fa.fai` is missing, build it. ```bash samtools faidx ref.fa ls -la ref.fa.fai ``` ## SAM File Validation Step by Step The ten steps below go from cheapest to most thorough. On a small file you can run all of them. On a 200 GB BAM you will usually run steps 1 through 4 routinely and reserve steps 5 through 8 for suspicious files or for release gates. ### Step 1: Confirm the file is what you think it is Extensions lie. A file named `sample.sam` may be a BAM, a gzip stream, or an empty placeholder left by a killed process. ```bash file sample.sam head -c 300 sample.sam htsfile sample.sam ``` Expected output for a genuine plain-text SAM: ``` sample.sam: ASCII text, with very long lines (65536), with no line terminators @HD VN:1.6 SO:coordinate @SQ SN:chr1 LN:248956422 @RG ID:grp1 SM:sample1 PL:ILLUMINA LB:lib1 A00123:8:H3V7LDSX2:1:1101:1000:1000 99 chr1 100 60 8M2I4M1D3M = 300 216 TTAGATAAAGGATACTG IIIIIIIIIIIIIIIII NM:i:3 RG:Z:grp1 ``` `htsfile` reports the detected format, for example `sample.sam: SAM version 1.6` or `sample.bam: BAM version 1.6`. If `file` reports `gzip compressed data`, you have a BAM or a gzipped SAM and the downstream commands need adjusting. A file that begins with binary bytes but has a `.sam` extension is a common artifact of `samtools view -b` without an output redirect. Rename it and move on. ### Step 2: Run the fast integrity check `samtools quickcheck` is the cheapest useful test. It verifies that the file exists and is readable, that the header is present and contains at least one target sequence, and, for BAM and CRAM only, that the end-of-file marker is present. ```bash samtools quickcheck -v sample.sam echo "exit code: $?" ``` A healthy file produces no output and an exit code of 0. The `-v` flag makes failures verbose. Typical failure text includes `sample.sam was missing EOF block` for a truncated BAM, or `sample.sam had no targets in header` for a SAM whose `@SQ` lines were stripped. For a directory of files, pass them all at once. `quickcheck` exits nonzero if any file fails, which makes it usable as a gate in shell scripts. ```bash samtools quickcheck -v *.sam *.bam ``` This step catches truncation and missing headers. It does not catch field-level errors, so do not stop here. ### Step 3: Inspect the header The header is where most semantic problems originate. Read it in full before parsing records. ```bash samtools view -H sample.sam ``` A complete, well-formed header looks like this: ``` @HD VN:1.6 SO:coordinate @SQ SN:chr1 LN:248956422 @SQ SN:chr2 LN:242193529 @RG ID:grp1 SM:sample1 PL:ILLUMINA LB:lib1 PU:UNIT1 @PG ID:bwa PN:bwa VN:0.7.17 CL:bwa mem -t 8 ref.fa r1.fq r2.fq @PG ID:samtools PN:samtools PP:bwa VN:1.20 CL:samtools sort -o sample.bam @CO validated 2026-11-10 ``` Check four things at this stage. 1. `@HD VN` is present and is a version you recognize (1.0 through 1.6 are in use). 2. Every contig that appears in column 3 of any record has a matching `@SQ SN` line, and the `LN` value matches the real contig length. 3. Every `RG:Z:` tag in the body has a matching `@RG ID:` line. 4. The `@PG` chain is intact, with each program's `PP` pointing at the previous `ID`. This is provenance, not correctness, but a broken chain is a signal that the file was assembled by hand. If the header is empty, you get `[E::sam_parse1] missing SAM header` from any downstream samtools command. ### Step 4: Force a full parse `samtools quickcheck` reads only the start and end of the file. To touch every record, decode the whole thing and discard the output. ```bash samtools view -c sample.sam ``` The command prints the number of alignment records, for example `12480033`. If any record fails to parse, samtools stops and prints an error. That is the point: a full parse is the cheapest syntactic validation that exists. To parse and re-encode without keeping output, which also exercises the BAM writer: ```bash samtools view -b -o /dev/null sample.sam ``` Three important caveats. - The parse stops at the first fatal error. A message about one bad record does not mean there is only one bad record. - `samtools view` does not check mate consistency, sort order claims, or tag correctness. - A file can parse perfectly and still be semantically wrong. Treat a clean full parse as a necessary condition, not a sufficient one. ### Step 5: Run Picard ValidateSamFile Picard's `ValidateSamFile` is the most thorough single validator available. It checks header structure, record field validity, mate information, sort order, read group references, CIGAR validity, and optional tag syntax. ```bash picard ValidateSamFile \ I=sample.sam \ MODE=SUMMARY \ VALIDATION_STRINGENCY=STRICT \ MAX_OUTPUT=100 ``` `MODE=SUMMARY` returns a count per error type. `MODE=VERBOSE` returns every offending record, which is what you want when debugging. `VALIDATION_STRINGENCY=STRICT` makes warnings fail the run. `MAX_OUTPUT` caps the volume of text so a single systemic problem does not produce a million lines. A clean result looks like this: ``` No errors found ``` A result with problems looks like this: ``` ERROR::MISSING_READ_GROUP 4231 ERROR::INVALID_CIGAR 12 WARNING::RECORD_MISSING_READ_GROUP 4231 ``` Read the error names literally. `MISSING_READ_GROUP` means the header has no `@RG` line but records carry `RG:Z:` tags, or the header has read groups but some records lack a tag. `INVALID_CIGAR` means a CIGAR string violated the grammar, for example an operator that consumes more query bases than SEQ contains. Picard also has a specific check worth knowing about: `MATE_NOT_FOUND` fires when a read claims a mate at coordinates where no such read exists. That class of error passes every samtools check and breaks tools that assume pairing. ### Step 6: Cross-check the reference dictionary This is the check that catches the most damaging class of silent error: a file aligned to one reference but validated against another. Contig names and lengths must agree exactly. ```bash cut -f1,2 ref.fa.fai | sort > ref_lengths.tsv samtools view -H sample.sam \ | awk -F'\t' '$1=="@SQ"{sn="";ln="";for(i=2;i<=NF;i++){if($i~/^SN:/)sn=substr($i,4);if($i~/^LN:/)ln=substr($i,4)};print sn"\t"ln}' \ | sort > sam_lengths.tsv comm -3 ref_lengths.tsv sam_lengths.tsv ``` `comm -3` prints lines unique to either file. Empty output means the dictionaries match. Any output means a contig is missing from one side or a length differs. A single mismatched length is a hard stop. A record at position 20,000,000 on a contig the SAM header declares as 19,000,000 bases long is an out-of-range coordinate, and tools will either crash or silently clamp. For CRAM files this step is not optional. CRAM stores sequence as differences against a reference, so the wrong reference produces wrong bases with no error at all. ```bash samtools view -T ref.fa -c sample.cram ``` If the reference is wrong or missing you will see `[E::cram_get_ref] Failed to fetch reference` or a checksum mismatch. ### Step 7: Check flags, pairing, and summary statistics `samtools flagstat` decodes the FLAG field across the whole file and reports the counts. It is a fast way to spot structural absurdities. ```bash samtools flagstat sample.sam ``` Representative output: ``` 1000000 + 0 in total (QC-passed reads + QC-failed reads) 0 + 0 secondary 0 + 0 supplementary 0 + 0 duplicates 950000 + 0 mapped (95.00% : N/A) 1000000 + 0 paired in sequencing 500000 + 0 read1 500000 + 0 read2 900000 + 0 properly paired (90.00% : N/A) 940000 + 0 with itself and mate mapped 10000 + 0 singletons (1.00% : N/A) 0 + 0 with mate mapped to a different chr 0 + 0 with mate mapped to a different chr (mapQ>=5) ``` Things to flag as suspicious: - `read1` and `read2` counts differ substantially in a paired-end library. - `properly paired` is far below `with itself and mate mapped` without an obvious biological reason. - `secondary` or `supplementary` counts are wildly out of line with the aligner used. - `QC-fail` counts are nonzero but the file was not filtered. `samtools stats` goes further, reporting insert size distribution, error rate, GC content, and per-cycle quality. ```bash samtools stats sample.sam > sample.stats.txt grep '^SN' sample.stats.txt | head -30 ``` The `SN` lines are the summary block. A mean insert size of zero on a paired-end library, or a raw error rate near 1, means something went badly wrong upstream. ### Step 8: Check CIGAR, SEQ, and QUAL arithmetic directly No off-the-shelf tool checks every arithmetic relationship, so write the check yourself. The rule is simple: the CIGAR operators that consume query bases (M, I, S, =, X) must sum to the length of SEQ, and the length of QUAL must equal the length of SEQ. | Op | Consumes query | Consumes reference | Meaning | |--|--|--|--| | M | yes | yes | alignment match or mismatch | | I | yes | no | insertion relative to reference | | D | no | yes | deletion from reference | | N | no | yes | skipped region, for example an intron | | S | yes | no | soft clip, bases present in SEQ | | H | no | no | hard clip, bases absent from SEQ | | P | no | no | padding | | = | yes | yes | sequence match | | X | yes | yes | sequence mismatch | ```python import sys import pysam path = sys.argv[4] mode = "r" if path.endswith(".sam") else "rb" QUERY_OPS = {0, 1, 4, 7, 8} # M I S = X REF_OPS = {0, 2, 3, 7, 8} # M D N = X checked = 0 bad_cigar = 0 bad_qual = 0 with pysam.AlignmentFile(path, mode) as fh: for rec in fh: checked += 1 if rec.cigartuples is None: continue qlen = sum(n for op, n in rec.cigartuples if op in QUERY_OPS) if rec.query_length is not None and qlen != rec.query_length: bad_cigar += 1 print(f"CIGAR/SEQ mismatch {rec.query_name} " f"cigar_query_len={qlen} seq_len={rec.query_length}") if rec.query_qualities is not None and len(rec.query_qualities) != rec.query_length: bad_qual += 1 print(f"QUAL/SEQ mismatch {rec.query_name} " f"qual_len={len(rec.query_qualities)} seq_len={rec.query_length}") print(f"checked={checked} cigar_mismatches={bad_cigar} qual_mismatches={bad_qual}") sys.exit(1 if (bad_cigar or bad_qual) else 0) ``` Run it and read the tail line. ```bash python check_cigar_seq.py sample.sam ``` ``` checked=1000000 cigar_mismatches=0 qual_mismatches=0 ``` Two subtleties matter here. Hard-clipped bases are not in SEQ, so H must be excluded from the query sum. Padding (P) consumes neither. Records with SEQ set to `*` legitimately have no sequence to check, and secondary or supplementary alignments sometimes omit SEQ to save space. To check the `NM` and `MD` tags against the reference, use `samtools calmd`. It recomputes both tags from the alignment and the reference. ```bash samtools calmd -b sample.sam ref.fa > /dev/null ``` Mismatches between the stored and recomputed tags produce warnings. This is a semantic check that catches stale tags after an edit. ### Step 9: Put the checks behind a single gate Individual commands are easy to forget. Wrap them so a pipeline fails loudly. ```bash #!/usr/bin/env bash set -euo pipefail sam="$1" ref="$2" samtools quickcheck -v "$sam" samtools view -c "$sam" > /dev/null samtools view -H "$sam" | grep -q '^@SQ' || { echo "no @SQ lines"; exit 1; } python check_cigar_seq.py "$sam" picard ValidateSamFile I="$sam" MODE=SUMMARY VALIDATION_STRINGENCY=STRICT MAX_OUTPUT=50 echo "VALIDATION PASSED: $sam" ``` Insert that script as the first step of every workflow that consumes alignments. A general-purpose validator inserted at pipeline boundaries is exactly the pattern PipeVal was designed to support, and the stated benefit is reduced compute wasted on corrupted files or invalid paths [1]. For format-specific batch work, look at how other bioinformatics formats solve the same problem. A validator built for neuron morphology files parses and corrects the syntactic structure of an entire directory of files, reports missing or invalid data values, returns basic statistics per file, and was benchmarked on thousands of files for runtime and efficacy [5]. The design is a good template: parse, report, correct where safe, summarize. SAM files have an analogous need, and writing a batch wrapper around the gate script above gives you the same shape. ### Step 10: Record what you did Validation without a record is not reproducible. Write the report alongside the data. ```bash { date -u samtools --version | head -2 picard ValidateSamFile --version 2>&1 | head -1 sha256sum sample.sam samtools quickcheck -v sample.sam && echo "quickcheck: PASS" samtools view -c sample.sam samtools flagstat sample.sam picard ValidateSamFile I=sample.sam MODE=SUMMARY VALIDATION_STRINGENCY=STRICT MAX_OUTPUT=200 } > sample.validation.txt 2>&1 ``` Store the report with the file in the archive. Formal validation procedures for file interpretation treat documentation of the tools and the process as part of the validation itself, not as an afterthought [2]. A checksum plus a tool version plus a pass or fail record lets a reviewer reproduce your result years later. ## Worked Example: Reading One Record and Checking the Arithmetic Take this single record: ``` r001 99 chr1 100 60 8M2I4M1D3M = 300 216 TTAGATAAAGGATACTG IIIIIIIIIIIIIIIII NM:i:3 RG:Z:grp1 ``` Check each field against the others. **FLAG 99.** Convert to binary: 99 = 64 + 32 + 2 + 1. Reading the bits: 0x1 paired, 0x2 proper pair, 0x20 mate reverse strand, 0x40 first in pair. The read is the first mate of a properly paired pair, and the mate maps to the reverse strand. That is consistent with RNEXT `=`, which means the mate is on the same contig. **CIGAR 8M2I4M1D3M.** Query-consuming operations are 8M + 2I + 4M + 3M = 17. Reference-consuming operations are 8M + 4M + 1D + 3M = 16. **SEQ.** `TTAGATAAAGGATACTG` is 17 characters. It matches the CIGAR query length. Good. **QUAL.** `IIIIIIIIIIIIIIIII` is 17 characters. In Phred+33, `I` is ASCII 73, so 73 minus 33 equals 40. Every base has Q40, a 1-in-10,000 error probability. Good. **Coordinates.** POS 100, reference span 16, so this read covers chr1:100-115. PNEXT is 300. If the mate also spans 16 bases, it covers chr1:300-315. TLEN is the rightmost mapped base of the pair minus the leftmost, plus one: 315 minus 100 plus 1 equals 216. The TLEN field says 216. Good. **NM tag.** `NM:i:3` claims three edit operations. With CIGAR 2I and 1D, that accounts for 3 non-match operations before counting any mismatches inside the M blocks. A value of 3 is therefore plausible only if all M blocks are perfect matches. `samtools calmd` will confirm or refute this. This is the whole discipline of SAM validation in miniature. Every field constrains another field, and a validator's job is to check those constraints systematically rather than trusting the file. ## Command Reference Table | Command | What it proves | Speed on 1M reads | Catches | |--|--|--|--| | `file` and `htsfile` | true format and encoding | instant | mislabeled files | | `samtools quickcheck -v` | header present, EOF block intact | instant | truncation, empty headers | | `samtools view -H` | header structure and dictionary | instant | missing `@SQ`, broken `@RG` | | `samtools view -c` | every record parses | seconds | malformed fields, bad CIGAR grammar | | `samtools view -b -o /dev/null` | parse plus re-encode | seconds | writer-level incompatibilities | | `picard ValidateSamFile` | full specification conformance | minutes | mate errors, sort order, tag syntax | | dictionary diff against `.fai` | reference identity | instant | wrong reference, wrong contig lengths | | `samtools flagstat` | FLAG distribution | seconds | pairing and mapping anomalies | | `samtools stats` | insert size, error rate, GC | seconds | library-level problems | | custom CIGAR/SEQ/QUAL script | arithmetic consistency | seconds | length mismatches Picard may miss | | `samtools calmd` | `NM` and `MD` correctness | minutes | stale tags after manual edits | ## Common Mistakes and Limitations **Treating a clean `samtools view` as a validation pass.** It is a parse. It stops at the first fatal record and it never checks mate consistency, sort order, or tag correctness. Files with thousands of semantically broken records pass this test every day. **Forgetting the `@SQ` lines.** A SAM file with alignments but no reference dictionary is not valid. It fails `quickcheck` with `had no targets in header` and fails Picard with a header error. This happens when someone extracts records with `grep` and drops the header lines. **Editing SAM with `sed` or `awk` and converting tabs to spaces.** The format is tab-delimited. Spaces produce `[E::sam_parse1] too few fields` on the very next parse. If you must edit by hand, use a tool that preserves tabs, or set `awk -F'\t' -v OFS='\t'` explicitly. **Trusting `@HD SO:coordinate` without checking.** The header is a claim, not a proof. Picard catches unsorted files that claim to be sorted. Tools that index BAM files will fail later with `[E::hts_idx_push] Unsorted positions`, which is a confusing message if you did not know the header was lying. **Ignoring warnings.** `RECORD_MISSING_READ_GROUP` is a warning, not an error, and it is fatal for RNA-seq quantification tools that aggregate by read group. Run with `VALIDATION_STRINGENCY=STRICT` for release builds. **Using the wrong reference with CRAM.** This produces wrong sequence with no error message. The dictionary check in step 6 is the only reliable guard. **Assuming one error means one bad record.** Parsers abort. The first error you see may be the ten-thousandth problem in the file. Fix the systematic cause, then re-run. **Validating once and never again.** Files get copied, concatenated, recompressed, and partially overwritten. Re-validate at each pipeline boundary, which is the pattern that general-purpose validators are designed to support [1]. **Skipping validation on files that "came from a trusted source."** Trusted sources produce truncated transfers, full disks, and interrupted writes. The checksum and the quickcheck cost seconds. **Confusing FASTQ and SAM validation.** A FASTQ file has no header dictionary, no CIGAR, and no mate fields. Tools and checks do not transfer between the two formats. Validate the FASTQ before alignment and the SAM after, with the right tool for each. **Overlooking batch-scale behavior.** A validator that works on one file may be unusable on ten thousand. Batch-oriented validators for other bioinformatics formats report per-file statistics and were explicitly benchmarked on thousands of files for runtime and efficacy, which is the right expectation to set for your own gate script [5]. **No record of the validation.** Without a stored report, a later reviewer cannot tell whether the file was validated, which tool version was used, or what the result was. Documentation is part of validation, not a separate administrative task [2]. **Assuming a standard exists where none does.** File format standards for alignment data are well defined, but conventions for what counts as "validated" vary by project. Frameworks in other data domains had to publish explicit recommendations covering file formats, input content, and reporting before cross-study comparison was possible [3]. Write down your own acceptance criteria and apply them uniformly. ### Limitations worth stating plainly Validation confirms internal consistency and conformance to the specification. It does not confirm that the alignment is biologically correct. A file can be perfectly valid and still contain reads mapped to the wrong reference, adapters that were never trimmed, or a library that was swapped at the sequencing center. Those problems require different checks: contamination screening, adapter content, insert size distribution, and concordance with known genotypes. Validation also cannot recover a file that is already corrupt beyond the point of a consistent record stream. If `quickcheck` reports a missing EOF block on a BAM, the truncated tail is gone. You can salvage the readable prefix with `samtools view` up to the failure point, but you cannot reconstruct the missing records. For any individual research or clinical dataset, the acceptance criteria and the interpretation of a failed check depend on the study design and should be set with the analyst or laboratory director responsible for the project. ## Frequently Asked Questions ### What is the fastest way to check a SAM file? Run `samtools quickcheck -v file.sam`. It returns nothing and exit code 0 for a file with an intact header, and it finishes in milliseconds regardless of file size. ### Does samtools view validate a SAM file? Only at the syntax level. `samtools view` parses every record and fails on malformed fields, but it does not check mate consistency, sort order claims, or tag correctness. Use Picard `ValidateSamFile` for a full audit. ### What is the difference between an ERROR and a WARNING in ValidateSamFile? An ERROR means the file violates the specification and downstream tools may fail. A WARNING means the file is technically legal but likely to cause problems, such as records missing a read group tag. ### Can a SAM file be valid and still be wrong? Yes. Validation checks internal consistency and format conformance. A file aligned to the wrong reference, or built from a mislabeled library, can pass every check and still produce meaningless results. ### How do I validate a BAM or CRAM file? The same commands work. `samtools quickcheck` additionally verifies the EOF block on BAM and CRAM, and CRAM validation requires the reference via `samtools view -T ref.fa`. ### Why does my SAM file fail with "missing SAM header"? The file has alignment records but no `@` header lines, usually because the header was stripped during a text edit or a `grep`. Prepend the correct header or regenerate the file from the aligner. ### Is FASTQ the same as SAM? No. FASTQ holds raw reads with per-base quality and no alignment information. SAM holds reads after alignment, with reference coordinates, CIGAR strings, flags, and mate information added. ### How do I check a SAM file without samtools? Use `pysam` in Python, which wraps the same htslib parser, or Picard `ValidateSamFile`, which is Java-based and independent of samtools. Any parser that conforms to the hts-specs definition will do. ## Further Reading - [Benchmarking Vision-based Analog Gauge Reading Methods for Vehicle Dashboard Validation](https://doi.org/10.1109/IAVVC61942.2025.11219595) ## Sources 1. [PipeVal: light-weight extensible tool for file validation](https://pubmed.ncbi.nlm.nih.gov/38341658/) 2. [Reliability validation for file system interpretation](https://doi.org/10.1016/j.fsidi.2021.301174) 3. [SzCORE: Seizure Community Open-Source Research Evaluation framework for the validation of electroencephalography-based automated seizure detection algorithms](https://pubmed.ncbi.nlm.nih.gov/39292446/) 4. [The Utility of the State Buy-In Variable in the Medicare Denominator File to Identify Dually Eligible Medicare-Medicaid Beneficiaries: A Validation Study](https://pubmed.ncbi.nlm.nih.gov/19840136/) 5. [Module for SWC neuron morphology file validation and correction enabled for high throughput batch processing](https://pubmed.ncbi.nlm.nih.gov/31971963/)