# The Role of Depth and Mapping Quality in Variant Filtering: Setting Thresholds That Balance Sensitivity and Precision ## Key Takeaways - Variant filtering decisions for sequencing depth (DP) and mapping quality (MQ) directly impact the balance between sensitivity (detecting true variants) and precision (avoiding false positives). Setting thresholds too high reduces sensitivity by discarding low-frequency or heterozygous variants, while setting them too low increases false positives from misaligned reads or sequencing artifacts. - Depth thresholds are critical for variant detection, with higher depth required for lower expected allele fractions. Germline heterozygous variants (approx. 50% allele fraction) typically require 10-30x DP for WGS/WES, while low-frequency somatic variants (<10% allele fraction) necessitate 30-100x DP to distinguish from sequencing errors. - Mapping quality (MQ) thresholds, typically 20-30, are essential for ensuring reads are correctly placed on the reference genome. Low MQ reads, common in repetitive regions or from paralogous sequences, can lead to spurious variant calls and require careful consideration, potentially through region-specific filtering or masking. - The relationship between depth and mapping quality is interactive; high depth can tolerate lower MQ, while marginal depth demands higher MQ for confidence. Thresholds should be evaluated jointly, considering sequencing platform, coverage uniformity (especially critical in WES), and variant type (SNVs vs. indels). - Empirical validation of thresholds using known variant sites is crucial. Examining coverage and mapping quality distributions, testing threshold combinations, and analyzing false positive candidates provide a data-driven approach to setting defensible filtering strategies that balance sensitivity and precision for specific study objectives. --- Variant filtering requires explicit decisions about sequencing depth and mapping quality that directly determine whether a candidate variant is reported or discarded. Depth thresholds control how many independent reads must support an allele call, while mapping quality thresholds control how confidently each read is placed at a genomic position. Setting these thresholds too high removes true low-frequency variants and reduces sensitivity. Setting them too low admits spurious calls from misaligned reads and sequencing artifacts, reducing precision. This article provides a systematic framework for selecting depth and mapping quality thresholds based on sequencing platform, coverage uniformity, and variant type, with worked examples from whole-genome sequencing (WGS) and whole-exome sequencing (WES). The practical outcome is a defensible filtering strategy that balances sensitivity for genuine variants against precision against false positives, supported by records that allow others to reproduce the decision process. ## Scope and Reader Context This guidance is written for biology students, researchers, laboratory professionals, and life-science practitioners who perform variant calling as part of genetic studies. The content assumes familiarity with basic sequencing concepts but does not require prior expertise in bioinformatics. The decisions described here apply to germline variant calling, where variants are inherited and expected to follow Mendelian patterns, and to somatic variant calling, where variants arise in individual tissues and may be present at low allele fractions. The filtering principles differ between these contexts, and the article addresses both. The central problem is that depth and mapping quality thresholds are often chosen by habit or default instead of by analysis of the data at hand. A threshold that works for one sequencing run may fail for another because of differences in library preparation, sequencing platform, read length, coverage distribution, and reference genome quality. The approach described here treats threshold selection as an empirical process grounded in the properties of the specific dataset. ## At a Glance: Threshold Decision Table The following table summarizes the key filtering parameters, their typical ranges, and the factors that should influence their selection. These values are starting points for investigation, not universal rules. | Parameter | Typical Starting Range | Primary Influence | Consequence of Setting Too High | Consequence of Setting Too Low | |---|---|---|---|---| | Minimum depth (DP) for germline WGS | 10 to 20 reads at a variant site | Expected coverage, platform error rate, variant type | Loss of true heterozygous calls in low-coverage regions | Increased false positives from sequencing errors | | Minimum depth (DP) for germline WES | 20 to 30 reads at a variant site | Capture efficiency, coverage uniformity across exons | Loss of variants in poorly captured exons | Increased false positives in GC-rich or repetitive regions | | Minimum depth (DP) for somatic calling | 30 to 100 reads at a variant site | Tumor purity, expected allele fraction, sequencing error rate | Loss of low-frequency somatic variants | False calls from sequencing artifacts | | Minimum mapping quality (MQ) | 20 to 30 | Reference genome quality, read length, presence of paralogs | Removal of valid reads from repetitive or duplicated regions | Misaligned reads support false variant calls | | Base quality threshold | 20 to 30 | Sequencing platform error profile | Removal of valid low-quality bases | Sequencing errors contribute to false calls | The relationship between depth and mapping quality is interactive. A site with very high depth can tolerate a lower mapping quality threshold because the multiplicity of reads provides confidence. A site with marginal depth requires higher mapping quality to ensure that the few supporting reads are correctly placed. Thresholds should be evaluated jointly, not independently. ## Core Principles of Variant Filtering ### Depth as a Measure of Evidence Sequencing depth, also called coverage, is the number of reads that align to a given genomic position. Depth provides the statistical foundation for variant calling because each read is an independent observation of the underlying DNA sequence. The probability that a true variant is observed at a site depends on the number of reads covering that site and the expected allele fraction of the variant. For germline heterozygous variants, the expected allele fraction is approximately 50 percent. At 10x depth, the probability of observing the variant allele in fewer than three reads is substantial, which means that sensitive germline calling requires either higher depth or a probabilistic model that accounts for the expected allele fraction. For somatic variants, allele fractions can be much lower, often below 10 percent, which requires substantially higher depth to achieve the same level of confidence. The relationship between depth and variant detection is not linear. Increasing depth from 10x to 20x provides a larger marginal benefit than increasing from 50x to 60x because the statistical confidence in an allele call grows with the square root of the number of observations. This diminishing return means that depth thresholds should be matched to the variant types of interest instead of set to the maximum achievable coverage. ### Mapping Quality as a Measure of Placement Confidence Mapping quality is a Phred-scaled probability that a read is incorrectly placed at a genomic location. A mapping quality of 20 corresponds to a 1 percent probability of misalignment, and a mapping quality of 30 corresponds to a 0.1 percent probability. Reads that align equally well to multiple genomic locations, such as reads from repetitive regions or gene families with high sequence similarity, receive low mapping quality scores. Low mapping quality reads are problematic for variant calling because they can support variants at the wrong genomic position. A read from a duplicated region may align to both copies with similar scores, and the aligner may place it at one location arbitrarily. If the two copies differ at a nucleotide position, the misaligned read creates a false variant call at the location where it was placed. Mapping quality thresholds are particularly important in regions of the genome with segmental duplications, pseudogenes, and recently duplicated gene families. These regions are common sources of false positive variant calls, and they require either higher mapping quality thresholds or explicit masking based on known problematic regions. ### The Sensitivity-Precision Tradeoff Sensitivity is the proportion of true variants that are detected. Precision is the proportion of detected variants that are true. These two metrics move in opposite directions as thresholds change. Raising depth and mapping quality thresholds increases precision by removing low-confidence calls but decreases sensitivity by discarding genuine variants that happen to have low support. Lowering thresholds increases sensitivity but admits more false positives. The appropriate balance depends on the downstream use of the variant calls. A study searching for rare disease-causing variants may prioritize sensitivity because missing a true variant has serious consequences. A study building a reference panel for imputation may prioritize precision because false variants in the panel propagate errors to all downstream analyses. The filtering strategy should be documented in terms of the sensitivity and precision goals for the specific study. ## Sequencing Platform and Coverage Considerations ### Whole-Genome Sequencing Coverage Patterns WGS aims to sequence the entire genome uniformly, but coverage is never perfectly uniform. GC content, repetitive elements, and local sequence complexity all influence how efficiently fragments are amplified and sequenced. The practical consequence is that some genomic regions fall below the average coverage, and these regions are exactly where variant calling becomes unreliable. A WGS dataset with 30x average coverage will have substantial fractions of the genome at 10x or below. These low-coverage regions are not randomly distributed. They cluster in GC-rich and GC-poor regions, near telomeres and centromeres, and in segmental duplications. A depth threshold that works for the genome average will discard true variants in these regions, and a threshold low enough to capture them will admit false positives elsewhere. The Ashkenazi Jewish reference panel study sequenced 738 samples at full depth of at least 30x across two platforms and developed quality control steps to optimize sensitivity, specificity, and comprehensiveness of variant calls. The study identified quality control thresholds that permitted accurate calling of single nucleotide variants across 94 percent of the genome, and it also identified regions that are poorly mapped using current reference assemblies. This finding illustrates that even at high depth, a fraction of the genome cannot be called reliably, and filtering decisions should account for these regions explicitly. ### Whole-Exome Sequencing Coverage Patterns WES enriches for protein-coding regions before sequencing, which changes the coverage distribution substantially. Capture efficiency varies across exons, and some exons are consistently under-captured because of their GC content or secondary structure. The result is that WES coverage is less uniform than WGS coverage, and depth thresholds must be set with this variability in mind. The practical implication is that a WES dataset with 80x average depth may have individual exons at 10x or below. These poorly captured exons are often the ones with biological relevance, and discarding them because of a uniform depth threshold can remove the variants most relevant to the study question. Exome-based studies should examine the coverage distribution across target regions before setting depth thresholds, and they should consider per-target filtering instead of a single genome-wide threshold. ### Low-Coverage Sequencing Applications Some study designs intentionally use low coverage to reduce cost. The indigenous African cattle study generated whole-genome sequence data for 240 animals with an average genome coverage of approximately 10x. The study identified approximately 43 million SNPs and 6 million indels, demonstrating that low-coverage sequencing can produce useful variant calls when the analysis pipeline is designed for it. Low-coverage variant calling requires different filtering strategies than high-coverage calling. Individual genotype calls at 10x depth are unreliable, but population-level variant discovery can still be accurate because the same variant is observed across multiple individuals. Low-coverage studies often use genotype likelihoods instead of hard genotype calls, and they may use imputation to fill in missing genotypes. The depth threshold for reporting a variant in a low-coverage study should be based on the cumulative evidence across all samples, not the depth in any single sample. ## Mapping Quality and Reference Genome Context ### Reference Genome Quality and Its Effect on Mapping The reference genome is the scaffold against which reads are aligned, and its quality directly affects mapping quality scores. A reference with gaps, misassemblies, or unresolved repetitive regions produces low mapping quality scores for reads that align to those regions, regardless of the sequencing quality. The Ashkenazi Jewish reference panel study identified numerous regions that are poorly mapped using current reference or alternate assemblies. These regions are not amenable to variant calling with any depth threshold because reads cannot be placed confidently. Studies should identify these regions in advance and exclude them from variant calling instead of attempting to rescue them with lower thresholds. The cattle study mapped reads to the ARS-UCD1.2 reference genome with a mean mapping rate of 99.2 percent, but the range across samples was 64.5 to 99.9 percent. This wide range indicates that some samples had substantial fractions of reads that could not be mapped, which would reduce effective coverage and require different filtering thresholds for those samples. ### Repetitive Regions and Paralogous Sequences Repetitive regions and paralogous gene families are the primary sources of low mapping quality reads. When a read sequence matches multiple genomic locations, the aligner assigns a low mapping quality because the placement is ambiguous. Variant callers may still use these reads, but the resulting calls are unreliable. The Quercus rubra mapping study used double digest restriction site associated DNA sequencing (ddRADseq) and found that the major driver of map inflation was multiple SNPs located within the same sequence, accounting for 77 percent of SNPs called. This finding illustrates that sequence context, beyond depth, determines whether a variant call is reliable. The study generated the highest quality map with a low level of missing data and a genome-wide threshold for deviation from Mendelian expectation, and the final map included only 1.8 percent of the SNPs initially called. For variant filtering, the practical implication is that mapping quality thresholds should be evaluated in the context of the specific genomic region. A mapping quality threshold of 20 may be adequate in unique regions but insufficient in paralogous regions. Some analysis pipelines apply region-specific thresholds or mask known problematic regions entirely. ### Read Length and Alignment Ambiguity Read length influences mapping quality because longer reads provide more sequence information for unique placement. Short reads, typically 100 to 150 base pairs in current Illumina platforms, may not span enough unique sequence to distinguish between closely related genomic locations. The cattle study used paired-end reads generated on the Illumina NovaSeq 6000 platform, which is typical of current WGS approaches. Paired-end reads help resolve mapping ambiguity because the two reads of a pair are expected to be at a known distance and orientation. A read pair that spans a repetitive region can be placed uniquely if one member of the pair falls in unique sequence. Variant callers that use paired-end information can achieve higher effective mapping quality than those that treat reads independently. ## Variant Type and Filtering Implications ### Single Nucleotide Variants SNVs are the simplest variant type and the most amenable to depth and mapping quality filtering. A single nucleotide change is supported by individual reads, and the evidence for the call is the number and quality of reads carrying the alternate allele. The cattle study identified approximately 43 million SNPs across 240 samples, and the Ashkenazi Jewish study demonstrated that quality control thresholds could permit accurate calling of SNVs across 94 percent of the genome. These studies show that SNV calling is reliable in most of the genome when depth and mapping quality thresholds are set appropriately. For SNV filtering, the key parameters are the minimum depth at the site, the minimum number of reads supporting the alternate allele, and the minimum mapping quality of the supporting reads. The alternate allele read count is often more informative than total depth because it directly measures the evidence for the variant. ### Insertions and Deletions Indels are more difficult to call than SNVs because they require the aligner to introduce gaps in the read alignment. The alignment around an indel is ambiguous, particularly in repetitive or homopolymer regions, and mapping quality scores for reads spanning indels are often lower than for reads in clean sequence. The cattle study identified approximately 6 million indels of 50 base pairs or less, and these indels were unevenly distributed across the genome. The uneven distribution reflects the difficulty of calling indels in certain sequence contexts. Indel filtering typically requires higher depth thresholds than SNV filtering because the alignment uncertainty creates more false positives. For indel filtering, the mapping quality of the reads spanning the indel breakpoints is more important than the mapping quality of the reads entirely within the indel. Some variant callers provide separate quality scores for indels, and these should be used in addition to depth and mapping quality thresholds. ### Somatic Variants and Low Allele Fractions Somatic variant calling differs from germline calling because the variant allele fraction is unknown and can be very low. A somatic variant present in a small fraction of tumor cells may have an allele fraction of 5 percent or less, which requires high depth to detect with confidence. The VariantMedium study developed a somatic variant caller that combines a tree-based classifier with a 3D densely connected convolutional network and evaluated it on experimentally confirmed variant data. The study used 336,839 variants from 2,956 samples with whole exome or genome sequencing for training and validation, and 118,887 variants from two independent studies with deep sequencing data for evaluation. The method showed the highest sensitivity among benchmarked callers and performed particularly well in genomic regions with high sequencing error rates. For somatic variant filtering, depth thresholds must be set based on the expected allele fraction and the sequencing error rate. A site with 100x depth can detect a 5 percent allele fraction variant with approximately five supporting reads, but the same variant at 30x depth would have only one or two supporting reads, which is indistinguishable from sequencing error. Somatic studies should also account for tumor purity, because the observed allele fraction is diluted by normal cells in the sample. ## Practical Workflow for Threshold Selection ### Step 1: Assess Coverage Distribution Before Filtering Before setting any thresholds, examine the coverage distribution across the dataset. Generate a histogram of depth across all callable sites, and calculate the fraction of the genome or exome target at various depth levels. This assessment establishes the baseline for threshold selection. For WGS, calculate the fraction of the genome at 10x, 20x, and 30x depth. For WES, calculate the fraction of target bases at 20x, 30x, and 50x depth. Compare these fractions to the study goals. If the study requires sensitive detection of heterozygous variants, the depth threshold should be set where the cumulative coverage curve is still high. The Quercus rubra study demonstrated the value of modeling lower depth scenarios. The study downsampled individual FASTQ files to model lower depth of coverage and found that sequencing the progeny using 96 samples per lane would have yielded too few SNP markers to generate a map, even with high parental depth. This approach of modeling the consequences of different depth levels before committing to a threshold is directly applicable to variant filtering. ### Step 2: Evaluate Mapping Quality Distribution Examine the distribution of mapping quality scores across the dataset. Identify the fraction of reads with mapping quality below 20, below 30, and below 40. High fractions of low mapping quality reads indicate either a problematic reference genome, short read lengths, or a sample with substantial contamination. Plot mapping quality against depth for known variant sites. This analysis reveals whether low mapping quality reads are concentrated in specific genomic regions. If they are, consider region-specific thresholds instead of a single genome-wide threshold. ### Step 3: Test Threshold Combinations on Known Variants If the dataset includes known variant sites, such as those from a genotyping array or a previously validated call set, test different threshold combinations against these sites. Calculate sensitivity as the proportion of known variants that pass the thresholds, and calculate precision as the proportion of passing calls that match known variants. This empirical validation is the most direct way to select thresholds. The Ashkenazi Jewish study used quality control steps to optimize sensitivity, specificity, and comprehensiveness of variant calls, and the resulting thresholds were validated by imputation accuracy. The same principle applies to any study with a reference call set. ### Step 4: Examine False Positive Candidates After applying initial thresholds, examine the variants that fail the thresholds. Determine whether they cluster in specific genomic regions, whether they have unusual allele fractions, or whether they are supported by reads with specific characteristics. This examination often reveals systematic issues that require threshold adjustments. Common false positive patterns include variants in homopolymer runs, variants near the ends of reads, variants in regions with high sequence similarity to other genomic locations, and variants supported by reads with low base quality. Each pattern suggests a specific filtering adjustment. ### Step 5: Document Thresholds and Rationale Record the final thresholds, the coverage and mapping quality distributions that informed them, and the sensitivity and precision estimates at the chosen thresholds. This documentation allows others to evaluate whether the thresholds are appropriate for their purposes and allows the study to be reproduced. Reproducibility is a core principle of bioinformatics practice. The [Galaxy Training Network](https://training.galaxyproject.org/) provides accessible workflow training and analysis tutorials that emphasize reproducible analysis practices, and the [nf-core documentation](https://nf-co.re/docs) describes community pipeline standards for reproducible workflow configuration. These resources support the documentation and sharing of analysis decisions. ## Records and Measurements for Threshold Decisions ### Coverage Statistics to Record Record the mean, median, and standard deviation of depth across the callable genome or exome target. Record the fraction of sites at key depth thresholds, including 10x, 20x, 30x, and 50x. Record the fraction of sites with zero depth, as these sites are excluded from variant calling entirely. For WES, record the fraction of target bases at each depth threshold separately from the genome-wide statistics. Capture efficiency varies by genomic region, and the target-specific statistics are more relevant for threshold decisions. ### Mapping Quality Statistics to Record Record the distribution of mapping quality scores across all aligned reads. Record the fraction of reads with mapping quality below 20, below 30, and below 40. Record the mean mapping quality for reads that support variant calls, and compare this to the mean mapping quality for all reads. Record the fraction of the genome or exome target with low mappability, defined as regions where reads cannot be placed uniquely. These regions should be excluded from variant calling or analyzed with separate thresholds. ### Variant Call Statistics to Record Record the total number of variant calls before filtering, after depth filtering, after mapping quality filtering, and after all filtering steps. Record the transition from each filtering step to the next, as this reveals which filters remove the most calls. Record the number of variants that pass filters but fail visual inspection or orthogonal validation. This number provides an estimate of the false positive rate at the chosen thresholds. ## Common Failure Patterns in Threshold Selection ### Using a Single Depth Threshold for All Variant Types A single depth threshold applied uniformly to SNVs, indels, and somatic variants will be wrong for at least one variant type. SNVs can be called reliably at lower depth than indels, and somatic variants require higher depth than germline variants. The threshold should be set separately for each variant type based on its specific error profile. ### Ignoring Coverage Uniformity Average depth is a poor guide for threshold selection because coverage is never uniform. A dataset with 30x average depth may have substantial regions at 5x, and a depth threshold of 10 will discard true variants in those regions. The coverage distribution, not the average, should drive threshold selection. ### Setting Mapping Quality Thresholds Without Examining the Reference Mapping quality scores are only meaningful in the context of the reference genome. A reference with unresolved repetitive regions produces low mapping quality scores in those regions regardless of read quality. Studies should identify problematic regions in the reference and account for them in threshold selection. ### Applying Germline Thresholds to Somatic Data Germline and somatic variant calling have different statistical foundations. Germline variants are expected at 50 percent or 100 percent allele fraction, while somatic variants can be at any allele fraction. Applying germline depth thresholds to somatic data will miss low-frequency somatic variants, and applying somatic thresholds to germline data will admit excessive false positives. ### Failing to Validate Thresholds on Known Variants Thresholds selected without validation on known variants are guesses. The sensitivity and precision of the chosen thresholds should be estimated empirically using a reference call set. The Quercus rubra study demonstrated the value of modeling the consequences of different depth levels, and the same approach applies to threshold validation. ## Limitations of Depth and Mapping Quality Filtering ### Regions That Cannot Be Resolved at Any Depth Some genomic regions cannot be called reliably at any depth because reads cannot be placed uniquely. These regions include centromeres, telomeres, and large segmental duplications. The Ashkenazi Jewish study identified numerous regions that are poorly mapped using current reference assemblies, and these regions were excluded from the high-quality call set. Studies should identify these regions in advance and exclude them from analysis instead of attempting to rescue them with lower thresholds. The fraction of the genome that is callable should be reported alongside the variant calls. ### Platform-Specific Error Profiles Different sequencing platforms have different error profiles, and these differences affect the appropriate thresholds. The Ashkenazi Jewish study sequenced samples on two platforms, Illumina X Ten and Complete Genomics, and developed platform-specific quality control thresholds. Thresholds validated on one platform should not be assumed to work on another. ### Batch Effects and Sample Quality Variation Sequencing quality varies across batches and across samples within a batch. The cattle study reported a mean mapping rate of 99.2 percent with a range of 64.5 to 99.9 percent across samples, indicating substantial sample-to-sample variation. Samples with low mapping rates require different thresholds than samples with high mapping rates. Studies should examine per-sample coverage and mapping quality statistics before applying uniform thresholds. Outlier samples should be identified and either excluded or analyzed with adjusted thresholds. ## Safety and Regulatory Context for Variant Filtering ### Clinical and Diagnostic Applications Variant filtering in clinical contexts carries regulatory and ethical obligations that extend beyond the technical considerations described here. Variant calls that inform medical decisions must meet higher standards of accuracy than research calls, and the filtering thresholds must be validated and documented accordingly. The [NCBI](https://www.ncbi.nlm.nih.gov/) provides official descriptions of databases, search systems, sequence resources, and analysis services that support clinical variant interpretation. Researchers working in clinical contexts should consult these resources and follow applicable regulatory requirements for variant calling and reporting. ### Data Sharing and Reproducibility Variant filtering decisions affect the interpretability of shared data. A variant call set generated with undocumented thresholds cannot be evaluated by downstream users, and it cannot be compared to other call sets. The [nf-core documentation](https://nf-co.re/docs) describes community pipeline standards that support reproducible workflow configuration, and the [Galaxy Training Network](https://training.galaxyproject.org/) provides accessible workflow training that emphasizes reproducible analysis practices. Studies should share the filtering thresholds, the coverage and mapping quality distributions that informed them, and the validation results. This documentation allows downstream users to evaluate whether the variant calls are appropriate for their purposes. ### Professional Escalation Criteria Researchers should escalate to a bioinformatics specialist or clinical genomics professional when the dataset shows patterns that cannot be resolved with standard filtering approaches. These patterns include a high fraction of reads with low mapping quality, a large proportion of the genome or exome target with zero depth, a substantial discrepancy between observed and expected variant counts, or a high rate of variant calls that fail visual inspection. The [Carpentries lessons](https://carpentries.org/lessons) provide foundational computing and data skills that can help researchers understand when they need specialized support. The [EMBL-EBI Training](https://www.ebi.ac.uk/training) program offers bioinformatics learning pathways and data-resource training that can build the skills needed to address complex filtering problems. ## A Decision Framework for Threshold Selection Based on Variant Type and Study Objective The preceding sections described the technical parameters that influence depth and mapping quality thresholds. This section provides a structured decision framework that translates those parameters into concrete filtering choices. The framework organizes threshold selection around three questions: what variant types matter for the study, what allele fractions are expected, and what downstream consequences follow from false positives versus false negatives. Answering these questions before examining the data prevents the common error of applying a generic threshold that fits neither the biological question nor the sequencing characteristics. ### Classifying Variants by Expected Allele Fraction The first step in the framework is to classify the variants of interest by their expected allele fraction. This classification determines the depth requirements before any data are examined. Germline heterozygous variants have an expected allele fraction near 50 percent, germline homozygous variants near 100 percent, and somatic variants anywhere from below 5 percent to near 50 percent depending on tumor purity and clonality. For germline studies, the allele fraction expectation is well defined by Mendelian inheritance. A depth threshold of 10 to 20 reads at a variant site provides reasonable confidence for heterozygous calls because the alternate allele is expected in roughly half the reads. The Quercus rubra mapping study demonstrated this principle in practice, using moderate sequencing depth of 15x in progeny to generate a high-quality genetic map. The study found that downsampling to lower depth would have yielded too few SNP markers to construct a map, establishing a practical lower bound for that application. For somatic studies, the expected allele fraction is the primary driver of depth requirements. A somatic variant present at 5 percent allele fraction requires substantially more depth than a germline heterozygous variant because the evidence for the alternate allele is diluted. The VariantMedium study, which developed a somatic variant caller trained on 336,839 variants from 2,956 samples, demonstrated that sensitivity for somatic variants depends critically on depth, particularly in genomic regions with high sequencing error rates. The study's performance was most pronounced in low-mappability regions, where depth alone cannot compensate for alignment ambiguity. ### Selecting Thresholds by Variant Type The framework applies different threshold combinations for SNVs, indels, and somatic variants. These variant types have different error profiles and therefore require different filtering stringency. For SNVs in germline WGS data, a depth threshold of 10 to 20 reads with a mapping quality threshold of 20 to 30 provides a reasonable starting point. The Ashkenazi Jewish reference panel study demonstrated that quality control thresholds could permit accurate calling of single nucleotide variants across 94 percent of the genome at full depth of at least 30x. The study's quality control steps optimized sensitivity, specificity, and comprehensiveness, and the resulting thresholds were validated by imputation accuracy. For indels, the framework recommends higher depth thresholds than for SNVs, typically 20 to 30 reads for germline WES and 15 to 25 for germline WGS. The higher threshold accounts for the alignment ambiguity around indel breakpoints. The cattle study identified approximately 6 million indels of 50 base pairs or less across 240 samples, and these indels were unevenly distributed across the genome. The uneven distribution reflects the difficulty of calling indels in certain sequence contexts, and the framework accounts for this by requiring more evidence for indel calls. For somatic variants, the framework recommends depth thresholds of 30 to 100 reads depending on the expected allele fraction and the sequencing error rate. A site with 100x depth can detect a 5 percent allele fraction variant with approximately five supporting reads, while the same variant at 30x depth would have only one or two supporting reads, which is indistinguishable from sequencing error. The VariantMedium study's approach of combining machine learning with experimental confirmation via targeted deep sequencing illustrates the depth requirements for confident somatic variant detection. ### Matching Thresholds to Study Objectives The framework explicitly links threshold selection to the downstream use of variant calls. Studies with different objectives require different sensitivity-precision balances, and the thresholds should reflect this. For discovery studies searching for rare disease-causing variants, the framework recommends lower depth thresholds to preserve sensitivity. Missing a true variant has serious consequences in this context, and the cost of additional false positives is acceptable because candidate variants will be validated by other methods. The Quercus rubra study exemplifies this approach, generating 78,725 SNP calls and then applying stringent filtering to retain only 849 high-quality markers for the final map. The initial permissive calling preserved sensitivity, while the downstream filtering achieved precision. For reference panel construction and imputation, the framework recommends higher thresholds to maximize precision. False variants in a reference panel propagate errors to all downstream analyses that use the panel. The Ashkenazi Jewish study demonstrated this principle, applying stringent quality control to produce a population-specific reference panel that produced more accurate imputation results than cosmopolitan panels, particularly for rare variants. The study's quality control steps were designed to optimize sensitivity, specificity, and comprehensiveness, with the balance weighted toward precision because of the panel's downstream use. For population genetics studies examining allele frequencies, the framework recommends thresholds that balance sensitivity and precision across the allele frequency spectrum. The cattle study, which sequenced 240 indigenous cattle at approximately 10x average coverage, identified approximately 43 million SNPs and 6 million indels. The study's approach of using population-level evidence instead of individual genotype confidence reflects the different threshold requirements for low-coverage population studies. ### Implementing the Framework in Practice The framework translates into a concrete implementation sequence. First, classify the variants of interest by type and expected allele fraction. Second, select initial depth and mapping quality thresholds based on the classification and the study objective. Third, examine the coverage and mapping quality distributions in the dataset to determine whether the initial thresholds are feasible. Fourth, validate the thresholds against known variant sites if available. Fifth, adjust the thresholds based on the validation results and document the final decisions. The implementation sequence should be recorded in a structured format that allows others to evaluate the decisions. The [Galaxy Training Network](https://training.galaxyproject.org/) provides accessible workflow training that emphasizes reproducible analysis practices, and the [nf-core documentation](https://nf-co.re/docs) describes community pipeline standards for reproducible workflow configuration. These resources support the documentation and sharing of threshold decisions. ### Recording Threshold Decisions and Rationale The framework requires that threshold decisions be recorded with their rationale. The record should include the variant types targeted, the expected allele fractions, the initial thresholds selected, the coverage and mapping quality distributions that informed the selection, the validation results, and the final thresholds applied. The record should also include the sensitivity and precision estimates at the chosen thresholds. These estimates provide a quantitative basis for evaluating whether the thresholds meet the study objectives. The [Bioconductor](https://bioconductor.org/) project provides official package and workflow documentation that supports reproducible genomic analysis, and the [EMBL-EBI Training](https://www.ebi.ac.uk/training) program offers bioinformatics learning pathways that can help researchers develop the skills needed to document and evaluate threshold decisions. ### Common Failure Patterns in Framework Application The framework addresses several common failure patterns in threshold selection. The first is applying a single threshold across all variant types without considering their different error profiles. The framework prevents this by requiring separate threshold selection for SNVs, indels, and somatic variants. The second failure pattern is selecting thresholds based on average coverage without examining the coverage distribution. The framework prevents this by requiring an examination of the coverage distribution before threshold selection. The cattle study's mapping rate range of 64.5 to 99.9 percent across samples illustrates why per-sample examination is necessary. The third failure pattern is applying germline thresholds to somatic data or vice versa. The framework prevents this by requiring classification of variants by expected allele fraction before threshold selection. The different depth requirements for germline and somatic variants are built into the framework's recommendations. The fourth failure pattern is failing to validate thresholds on known variants. The framework requires validation when a reference call set is available. The Quercus rubra study's approach of modeling the consequences of different depth levels through downsampling demonstrates the value of empirical validation. ### Escalation Criteria Within the Framework The framework includes explicit escalation criteria for situations where standard threshold selection is insufficient. Escalate to a bioinformatics specialist when the coverage distribution shows a large fraction of the genome or exome target below 10x depth, when mapping quality scores are consistently low across the dataset, when validation against known variants shows poor sensitivity or precision at any reasonable threshold, or when the variant calls show patterns that suggest systematic artifacts. The [NCBI](https://www.ncbi.nlm.nih.gov/) provides official descriptions of databases, search systems, sequence resources, and analysis services that can support troubleshooting. The [Carpentries lessons](https://carpentries.org/lessons) provide foundational computing and data skills that can help researchers understand when they need specialized support. These resources support the escalation process by providing the background knowledge needed to communicate effectively with specialists. ## Frequently Asked Questions ### What is the minimum depth needed to call a heterozygous germline variant? The minimum depth depends on the confidence level required and the sequencing error rate. At 10x depth, a heterozygous variant is expected to be observed in approximately five reads, which provides limited statistical confidence. At 20x depth, the expected support is approximately ten reads, which is more reliable. The appropriate threshold should be validated empirically using known variant sites in the dataset. ### How does mapping quality differ from base quality? Mapping quality is the probability that a read is placed at the correct genomic location, while base quality is the probability that a specific base call within a read is correct. A read can have high base quality but low mapping quality if it aligns ambiguously to multiple locations. Both metrics contribute to variant call confidence, but they measure different sources of error. ### Why do somatic variant calls require higher depth than germline calls? Somatic variants can be present at low allele fractions because they occur in a subset of cells within a tissue sample. A variant present in 10 percent of cells has an expected allele fraction of approximately 5 percent after accounting for the normal cells in the sample. Detecting a 5 percent allele fraction variant requires substantially more depth than detecting a 50 percent allele fraction germline variant. ### Should I use the same depth threshold for SNVs and indels? Indels generally require higher depth thresholds than SNVs because the alignment around indels is more ambiguous. Reads spanning an indel breakpoint may be misaligned, and the error rate for indel calls is higher than for SNV calls. The threshold should be set separately for each variant type based on its specific error profile. ### How do I handle regions with low mappability in my variant calls? Regions with low mappability should be identified and either excluded from variant calling or analyzed with separate thresholds. Reads in these regions cannot be placed uniquely, and variant calls from these regions are unreliable regardless of depth. The fraction of the genome or exome target with low mappability should be reported alongside the variant calls. ### What is the relationship between depth and the detection of rare variants? The probability of detecting a rare variant depends on the allele fraction and the depth at the variant site. A variant present at 1 percent allele fraction requires approximately 100x depth to be observed in a single read, and much higher depth for confident detection. Studies focused on rare variants should set depth thresholds accordingly. ### How do I validate my filtering thresholds? Validation requires a set of known variant sites, such as those from a genotyping array or a previously validated call set. Apply the candidate thresholds to the known sites and calculate sensitivity as the proportion of known variants that pass the thresholds. Examine the variants that fail the thresholds to identify systematic issues. ### When should I escalate to a bioinformatics specialist? Escalate when the dataset shows patterns that cannot be resolved with standard filtering approaches, such as a high fraction of reads with low mapping quality, a large proportion of the genome with zero depth, or a substantial discrepancy between observed and expected variant counts. These patterns may indicate problems with the sequencing run, the reference genome, or the analysis pipeline that require specialized expertise. ## Related Bioinformatics Guides - [Detecting Structural Variants with Long-Read Sequencing: Methods and Considerations](/knowledge/bioinformatics/detecting-structural-variants-with-long-read-sequencing-methods-and-considerations) - [Single-Cell RNA Sequencing Quality Control: A Practical Guide to Filtering and Metrics](/knowledge/bioinformatics/single-cell-rna-sequencing-quality-control-a-practical-guide-to-filtering-and-metrics) - [From Raw Reads to Variants: A Diagnostic Blueprint for Next-Generation Sequencing (NGS) Workflows](/knowledge/bioinformatics/ngs-raw-reads-variant-calling-blueprint) - [Genomic Data Analysis Tools: A Comparative Guide for Researchers](/knowledge/bioinformatics/genomic-data-analysis-tools-a-comparative-guide-for-researchers) - [How to Interpret Gene Set Enrichment Analysis Results](/knowledge/bioinformatics/how-to-interpret-gene-set-enrichment-analysis-results) ## Related Clinical & Scientific Guides * [A Practical Guide to Detecting Antimicrobial Resistance Genes in Shotgun Metagenomic Data](/knowledge/bioinformatics/a-practical-guide-to-detecting-antimicrobial-resistance-genes-in-shotgun-metagenomic-data) * [Computational Immunology: Modeling the Immune System](/knowledge/bioinformatics/computational-immunology-modeling-the-immune-system) * [How to Set Hard Filters for Germline Variant Calling: A Practical Guide to GATK Best Practices](/knowledge/bioinformatics/how-to-set-hard-filters-for-germline-variant-calling-a-practical-guide-to-gatk-best-practices) ## References and Further Reading - [NCBI Data Resources](https://www.ncbi.nlm.nih.gov/). National Center for Biotechnology Information. - [EMBL-EBI Training](https://www.ebi.ac.uk/training). European Bioinformatics Institute. - [Bioconductor](https://bioconductor.org/). Bioconductor Project. - [Galaxy Training Network](https://training.galaxyproject.org/). Galaxy Project. - [nf-core Documentation](https://nf-co.re/docs). nf-core. - [The Carpentries Lessons](https://carpentries.org/lessons). The Carpentries. - [High-quality genetic mapping with ddRADseq in the non-model tree Quercus rubra.](https://pubmed.ncbi.nlm.nih.gov/28558688). BMC genomics, 2017. - [High-depth whole genome sequencing of an Ashkenazi Jewish reference panel: enhancing sensitivity, accuracy, and imputation.](https://pubmed.ncbi.nlm.nih.gov/29705978). Human genetics, 2018. - [Whole-genome sequences of 240 indigenous African cattle from Egypt, Uganda, and South Africa.](https://doi.org/10.1038/s41597-026-07458-y). 2026. - [VariantMedium: sensitive and generalizable somatic point mutation calling with 3D DenseNets trained and evaluated on experimental data.](https://doi.org/10.1186/s13073-026-01675-1). 2026. > This article is educational and does not replace validated analysis plans, institutional policy, clinical interpretation, or specialist review.