Germline Variant Filtering

If you are a bioinformatician, clinical genomicist, or researcher analyzing human germline sequencing data and need to separate true inherited variants from sequencing artifacts and alignment errors, this guide is for you. Germline variant filtering is the systematic process of retaining high-confidence variant calls while removing false positives introduced during library preparation, sequencing, mapping, and initial calling. The goal is to produce a clean set of variants suitable for downstream interpretation, association studies, or clinical reporting. This guide provides a source-bounded framework covering core concepts, decision points, a practical workflow, quality checks, common mistakes, and limits of interpretation. NCBI Bookshelf offers authoritative background on variant calling and filtering strategies, and EMBL-EBI Training provides structured materials for implementing these methods in practice.

Germline variant filtering typically begins after joint calling or single-sample variant detection and includes steps such as hard filtering based on annotated quality metrics, application of machine learning derived quality scores (e.g., VQSR), and population frequency based annotation. The decision flow depends on sequencing depth, library complexity, and the intended use of the variant list. Understanding the logic behind each filter helps you avoid discarding true variants or retaining systematic errors.

At a Glance

Core Concepts

Germline variant filtering rests on several foundational ideas. First, sequencing artifacts produce characteristic signatures that differ from true genetic variation. For example, strand bias (where alternate alleles appear predominantly on one strand) often indicates mapping or PCR artifacts, while low variant allele fraction (VAF) may suggest contamination or early somatic events. Galaxy Training Network provides hands on workflows that illustrate how these metrics behave across real datasets.

Second, the quality metrics calculated by variant callers are not independent. A variant with borderline quality by depth (QD) but excellent mapping quality might be retained, while a variant with poor QD and strong strand bias is likely false. Hard filtering applies universal thresholds, but those thresholds must be tuned to your sequencing chemistry and depth. A common starting point for human WGS at 30x depth uses QD > 2.0, FS < 60, SOR < 3, MQ > 40, and ReadPosRankSum > -8.0 for SNPs, with adjusted values for indels. Bioconductor packages such as VariantAnnotation and GATK enable programmatic inspection and filtering of these metrics.

Third, machine learning based recalibration (VQSR) uses a training set of known true variants (e.g., from HapMap or Omni) to model the distribution of good versus bad variants and assign a log odds ratio. This method can outperform hard filtering when a large training resource is available, but it requires sufficient variant density and is less effective for small target panels. NCBI Sequence Read Archive hosts thousands of germline datasets that can be used to evaluate filtering performance, though you should always validate with truth sets from your own sample type.

Decision Points

Before writing a filtering pipeline, consider these decision points.

Sequencing depth and uniformity. Low coverage regions (below 10x) produce unreliable genotype calls. You must decide whether to keep variants with low depth or filter them. Retaining low depth variants increases false positives, while filtering may remove real calls in hard to sequence areas. A comprehensive review of variant calling tools for RNA seq (PubMed 42081940) highlights how depth thresholds differ between DNA and RNA based analyses, but the principle applies to germline DNA as well.

Library complexity and PCR duplicates. High duplicate rates inflate depth without adding independent observations. Use tools like Picard MarkDuplicates before variant calling. If duplicates remain in the alignments, consider filtering variants with high depth that are supported by few unique read pairs. Tumor naive ctDNA detection with deep learning enhanced error suppression (PubMed 42332784) demonstrates how controlling for duplicate artifacts improves sensitivity in challenging contexts, a lesson directly transferable to germline work.

Population frequency and inheritance model. For rare disease studies, common variants (e.g., MAF > 1% in gnomAD) are usually filtered out unless they show incomplete penetrance or recessive inheritance. For population genetics, common variants are the signal. Define your population frequency threshold based on the disease prevalence and cohort ancestry. Rare variants in inborn errors of immunity genes in young adults with severe COVID 19 (PubMed 42190421) illustrates how filtering by population frequency can isolate potentially causal rare variants.

Variant caller output specifics. GATK 4.x guidelines differ from those for DeepVariant or Strelka2. DeepVariant produces fewer systematic false positives but may still require hard filters for indels. Use the recommended filtering practices from the caller documentation or community best practices. MuSA a Nextflow pipeline for deep reproducible annotation and clinical ranking of genomic variants (PubMed 42304195) provides an example of how to integrate filtering steps within a reproducible pipeline that handles multiple callers.

Practical Workflow

The following workflow is a template for germline variant filtering. Adjust parameters based on your coverage and caller.

1. Prepare the raw VCF. Ensure your VCF has complete annotation fields. For GATK, run VariantAnnotator to add missing fields if necessary. For DeepVariant, the output VCF includes standard INFO fields.
2. Apply hard filters. Use bcftools filter or GATK VariantFiltration. Example for SNPs: QD < 2.0 | FS > 60 | SOR > 3 | MQ < 40 | MQRankSum < -12.5 | ReadPosRankSum < -8.0. For indels: QD < 2.0 | FS > 200 | SOR > 10 | MQ < 40 | ReadPosRankSum < -20.0. These thresholds are initial recommendations, examine metric distributions for your data using bcftools query and plot in R.
3. Check strand bias manually. Strand bias metrics (FS and SOR) are sensitive to strand specific errors. If your library preparation is directional (e.g., stranded RNA seq), adjust thresholds accordingly. For most WGS and WES, default values work.
4. Filter by read depth. Remove sites where total depth across all samples is unusually low (< 10) or extremely high (> 2.5x mean depth), as high depth often indicates collapsed repeats or paralogous sequences.
5. Apply genotype level filters. Use bcftools filter with --include 'FMT/DP > 10 & FMT/GQ > 20' to remove low confidence genotype calls per sample.
6. Remove clustered variants. False positive variants often appear in clusters due to alignment artifacts near indels or repeats. Use bcftools filter with --SnpGap 10 to remove isolated SNPs within 10 bp of an indel, or use a sliding window filter.
7. Apply VQSR (optional but recommended for WGS). For GATK, run VariantRecalibrator with resources (HapMap, Omni, 1000 Genomes, dbSNP) then ApplyVQSR with a truth sensitivity level (e.g., 99.9 for highly sensitive, 99.0 for more specific). For DeepVariant, consider using the provided quality scores without VQSR.
8. Annotate with population frequencies. Use bcftools annotate or VEP to add gnomAD AF. Then filter: bcftools filter --exclude 'INFO/gnomAD_AF > 0.01' for rare variant studies. Keep in mind that gnomAD contains some pathogenic variants, so consider using a filter like gnomAD_AF > 0.05 for common variant removal in dominant disease studies.
9. Final review. Check the ratio of transitions to transversions (Ti/Tv). For human WGS, expected Ti/Tv is around 2.0 to 2.1 for whole genome, 3.0 to 3.3 for exome. A lower ratio indicates many false positives. Also check heterozygote to homozygote ratio, which should be near 1 for common variants and higher for rare ones. Whole exome sequencing reveals rare DNA repair gene variants in BRCA1 2 negative Arab early onset breast cancer patients (PubMed 42098349) used such quality metrics to validate their filtering pipeline.

Quality Checks

After filtering, run three quality checks.

Truth set concordance. Download the Genome in a Bottle (GIAB) high confidence BED for your sample type (NA12878 for WGS, NA24385 for WES). Use bcftools isec to compare your filtered VCF with the truth set. Calculate precision (TP / (TP + FP)) and recall (TP / (TP + FN)). Aim for precision > 0.95 and recall > 0.90 for standard applications. Mechanism of Mutation in G Protein Gated Inwardly Rectifying K+ Channel in Familial Hyperaldosteronism Type III (PubMed 42280146) describes how missense variant interpretation relies on high confidence filtering, emphasizing the downstream impact of filtering quality.

Segregation analysis. For family samples (trio or larger pedigrees), verify that variants segregate according to Mendelian expectations. A high rate of Mendelian violations indicates either sequencing errors or filtering mistakes.

Visual inspection. Use IGV to inspect a random subset of filtered variants in the region of interest. Check for read support, mapping quality, and strand balance. A variant called from only two reads on one strand with high base quality might still be real if the read depth is low, but if the region has 30x coverage, that variant is suspicious.

Common Mistakes

Using universal thresholds without exploration. Hard filter values derived from the GATK best practices are starting points, not absolutes. If your data has lower depth, you may need to lower QD threshold. Plot the distribution of each metric before deciding cutoff.

Applying the same filters to SNPs and indels. Indel quality metrics have different expected distributions. For example, FS values for indels are often higher than for SNPs. Using SNP thresholds on indels will eliminate many true indels. Always filter SNP and indel separately.

Ignoring depth in low complexity regions. Homopolymer runs and simple repeats produce false indels. Hard filtering by allelic balance (AB > 0.2 for heterozygous) can help, but the best practice is to use a dedicated tool like GATK FilterMutectCalls (for somatic) or simply exclude variants in known repeat regions using the UCSC repeat masker BED.

Over filtering based on population frequency. A variant that is common in gnomAD may still be pathogenic in specific ancestries or for recessive disorders with founder effects. Use population frequency as a suggestion, not a rigid cutoff.

Failing to document filtering parameters. For reproducibility, record every filter and threshold in a pipeline log or a methods section. Future reanalysis or paper revisions require knowing exactly what was removed or retained.

Limits of Interpretation

Germline variant filtering cannot remove all errors. Systematic artifacts from GC bias, mappability issues, and repetitive elements will survive even stringent filters. Low coverage regions will always have higher false positive and false negative rates. Variant calling in segmental duplications, pseudogenes, and the MHC locus is notoriously unreliable regardless of filtering. For clinical applications, any variant with therapeutic or diagnostic implications should be validated by an orthogonal method such as Sanger sequencing or an independent sequencing platform.

Filtering is also sensitive to the choice of reference genome. A variant that passes filters with GRCh38 may fail with T2T CHM13 due to different annotation of repetitive elements. Always state the reference version and the filtering criteria in your reports. Finally, filters optimized for one population may perform poorly in another due to differences in linkage disequilibrium and variant density. Rare variants in inborn errors of immunity genes in young adults with severe COVID 19 (PubMed 42190421) found that filtering strategies transfer across cohorts only when accounting for ancestry specific allele frequencies, a caution applicable to all germline studies.

Frequently Asked Questions

1. Should I use hard filtering or VQSR? Hard filtering is simpler and works well for targeted panels and exomes with 50x or more depth. VQSR requires a large training set and is recommended for whole genome data with at least 30x coverage. If you lack the resource files, hard filtering with tuned thresholds performs adequately.

2. What does the strand bias metric FS mean? FS is Fisher's exact test p value for strand bias, transformed on a phred scale. A high FS value indicates the variant is seen mostly on one strand, suggesting an artifact. For SNPs, FS above 60 is usually problematic. For indels, thresholds up to 200 are often used because indels naturally show more strand bias.

3. How do I choose genotype filtering thresholds for DP and GQ? Depends on your sequencing depth. A common rule is minimum DP = 10 and minimum GQ = 20 for 30x WGS. For higher depth, DP can be set higher (e.g., 20) to avoid errors in low coverage regions. GQ is computed by the caller and reflects the confidence in the genotype assignment, values below 20 indicate uncertain calls.

4. Can I reuse a filtering pipeline from one study in another? Reusing a pipeline is acceptable as a starting point, but you must recalibrate thresholds based on the new data's coverage, library complexity, and variant caller. Pipelines from published studies often omit details about custom parameter adjustments, so always validate with truth sets or internal controls.

References and Further Reading

• NCBI Bookshelf , Authoritative chapters on variant calling and filtering in human genomics.
• EMBL-EBI Training , Online courses on variant analysis with practical exercises.
• Galaxy Training Network , Hands on workflows for germline variant filtering using standard tools.
• Bioconductor , R packages (VariantAnnotation, GATK) for programmatic filtering and visualization.
• NCBI Sequence Read Archive , Public repository for raw sequencing data used to test filtering pipelines.
• Tumor naive ctDNA detection with deep learning enhanced error suppression for sensitive mutation calling (PubMed 42332784) , Error suppression methods applicable to germline low frequency variant detection.
• MuSA a Nextflow pipeline for deep reproducible annotation and clinical ranking of genomic variants (PubMed 42304195) , Example of an integrated filtering and annotation pipeline.
• Rare variants in inborn errors of immunity genes in young adults with severe COVID 19 (PubMed 42190421) , Study demonstrating population frequency filtering in practice.
• Whole exome sequencing reveals rare DNA repair gene variants in BRCA1 2 negative Arab early onset breast cancer patients (PubMed 42098349) , Quality metrics and filtering used in a rare variant discovery study.
• A comprehensive review of variant calling tools for RNA seq: Challenges and advances (PubMed 42081940) , Discusses depth and filtering differences, relevant to germline DNA as well.

Related Articles

• Read Duplication Sequencing
• Per Base Quality Sequencing
• Kmer Spectrum Analysis
• Genome Size Estimation Kmer
• Contamination Screening Genome Assembly