Variant Recalibration

Variant recalibration is a statistical filtering step that separates true genetic variants from sequencing artifacts by modeling the distribution of variant annotations in a high confidence training set. If you are a bioinformatician, clinical researcher, or genome analyst working with high throughput sequencing data, this guide will help you understand when and how to apply recalibration, what decisions affect its performance, how to execute a practical workflow, and how to interpret results within their inherent limits.

Before jumping into the details, you should know that recalibration is not a universal fix. It works best with large cohorts and a reliable truth set. For smaller studies or targeted panels, hard filtering often performs better. The Galaxy Training Network provides excellent hands on tutorials that mirror many of the concepts discussed here. And the EMBL-EBI Training resources offer a thorough background on the underlying statistical principles.

At a Glance

Decision Criteria

You should consider variant recalibration when your dataset contains more than about 30 samples and you have access to a high quality truth set like the one provided by the NCBI Sequence Read Archive for reference populations. The following criteria help you decide:

• Cohort size. Recalibration relies on many variants to estimate model parameters. With fewer than 30 samples, the annotation distribution becomes sparse and the model may not converge reliably.
• Sequencing depth and uniformity. Recalibration assumes that depth related annotations such as DP and QD behave consistently across the genome. If your data has extreme coverage variation, consider additional normalization.
• Availability of orthogonal truth data. Training sets must be derived from independent genotyping arrays or deeply sequenced trios. Without a reliable truth set, recalibration can introduce systematic bias.
• Study design. For rare variant discovery or family based studies, hard filtering with carefully chosen thresholds is often safer.
• Computational resources. Model building and scoring are memory intensive. Evaluate your cluster or cloud capacity before starting.

A practical framework for this decision appears in the Bioconductor documentation for VariantTools and related packages, which include utilities to compare hard filtering and recalibration outcomes on your own data.

Workflow or Implementation Steps

Below is a generalized workflow that follows the GATK VQSR model. These steps can be adapted to other tools.

1. Prepare training sets and known sites. Download public resources such as HapMap, Omni, and 1000 Genomes high confidence SNPs and indels. Index them with tabix and ensure they match your reference genome build.

2. Run initial variant calling. Perform joint genotyping across all samples to produce a raw multisample VCF. The NCBI Bookshelf chapter on variant discovery covers this process in detail.

3. Select annotations for modeling. Common annotations include Quality by Depth (QD), Fisher Strand (FS), Mapping Quality (MQ), Mapping Quality Rank Sum, Read Pos Rank Sum, and Strand Odds Ratio. Remove highly correlated annotations to reduce model complexity. Use a correlation matrix or principal component analysis to guide this selection.

4. Build the recalibration model. For SNPs, gather the training sets and assign prior probabilities (eg. HapMap sites get a high prior, Omni sites a medium prior, and known non polymorphic sites may be excluded). Run VariantRecalibrator in GATK. Monitor convergence diagnostics: the number of iterations, log likelihood, and the number of variants in each cluster.

5. Evaluate the model output. Inspect the Gaussian mixture model parameters. True variants should cluster in a high quality region. Poor clustering with overlapping Gaussian components suggests the annotations are not discriminative enough or the training set is contaminated.

6. Apply the recalibration. Use ApplyVQSR to assign a VQSLOD tranche threshold. A common starting point is 99.9% sensitivity for SNPs and 99% for indels, but adjust based on your tolerance for false positives versus false negatives.

7. Review the filtered VCF. Count the number of variants before and after filtering. Check the transition transversion ratio (Ti/Tv) for expected ranges (around 2.0 to 2.1 for whole genome, 3.0 to 3.2 for exome) as a quality measure.

8. Optional iterative recalibration. Some pipelines run VQSR a second time using the filtered variants as a new training set. This can remove remaining artifacts but risks circular logic.

Quality Checks

After running recalibration, perform these standard checks:

• VQSLOD distribution. Plot histograms of VQSLOD for filtered and passed variants. A clear separation between the two groups indicates a well performing model.
• Ti/Tv ratio. Compare the ratio before and after filtering. A ratio that drops too low suggests over filtering of true variants.
• Genotype concordance. If you have array data for a subset of samples, calculate discordance between called genotypes and array genotypes. The Short Read Sequencing Benchmarking with Donor Specific Assemblies study provides a comparison framework that can be adapted for this purpose.
• Per sample metrics. Check the number of variants per sample after filtering. A sample that loses an unusually high number of variants may have poor sequencing quality that was not captured by the model.

Common Mistakes

• Using too few samples. Models trained on fewer than 30 samples often fail to converge. In such cases, stick to hard filtering.
• Including low quality training data. If the truth sets are from a different population or sequencing technology, the model will learn the wrong annotation distributions.
• Ignoring annotation correlation. Highly correlated annotations cause collinearity and unstable parameter estimates. Remove one of each correlated pair.
• Applying the same threshold to SNPs and indels. Indel annotations behave differently. Always train separate recalibration models for each variant type.
• Neglecting to check for batch effects. If samples were sequenced in different batches, the annotation space may be non uniform. Consider stratifying by batch or using batch specific truth sets.

A discussion of these pitfalls can be found in the EMBL EBI Training materials on variant filtering.

Limits of Interpretation

Variant recalibration has several inherent limitations that affect how you interpret its output.

• Overconfident scores. VQSLOD values are probabilities under the specified model, but they do not guarantee correctness. A variant with high VQSLOD can still be a false positive, especially if it lies in a poorly modeled region like repetitive sequences.
• Dependence on training set completeness. Recalibration cannot recognize novel variant classes that are absent from the training set. Rare variants with unusual annotation profiles may be incorrectly filtered.
• Sensitivity to annotation choice. If the chosen annotations do not separate truth from error on your specific platform, the model will underperform. For example, RNA editing sites may require different annotations because their base quality patterns differ from DNA variants.
• Computational instability. The Gaussian mixture model is initialized randomly. Different runs on the same data can yield slightly different cluster assignments. Set a random seed to make the results reproducible.
• No guarantee of improved downstream results. Even a perfectly tuned VQSR model can produce a VCF that leads to false biological conclusions if the underlying variant calling is biased.

The application of recalibration to specialized data types, such as low frequency variant detection, is addressed in recent works like the High precision Detection of RNA Editing Sites which highlights the need for tailored annotation sets.

Frequently Asked Questions

1. Can I use VQSR with fewer than 30 samples?

Technically yes, but it is strongly discouraged. With small cohorts the model cannot reliably separate signal from noise. Hard filtering with precise thresholds is a better alternative.

2. How do I choose the tranche sensitivity threshold?

There is no universal threshold. For rare variant discovery you may want a more permissive threshold (99.9% sensitivity) to avoid losing true positives. For clinical reporting you might use a stricter threshold (99.0%) to minimize false positives. Pilot experiments on your own data can guide the choice.

3. What is the difference between VQSR and hard filtering?

VQSR uses a machine learning model to adapt thresholds based on annotation distributions. Hard filtering applies fixed values across all variants. VQSR generally provides better sensitivity at a given specificity when the model assumptions hold, but it is more complex and requires larger sample sizes.

4. Do I need to recalibrate indels separately from SNPs?

Yes. Indels and SNPs occupy different annotation spaces. Joint recalibration can produce biased results. Always run separate models for each type and apply the appropriate filter to your VCF using the --filterName parameter.

References and Further Reading

• VQSR methodology overview: Galaxy Training Network
• Statistical foundation for variant filtering: NCBI Bookshelf
• Practical tutorial on GATK recalibration: EMBL EBI Training
• R based recalibration alternatives: Bioconductor
• Public truth sets for training: NCBI Sequence Read Archive
• Benchmarking variant calling pipelines: Short Read Sequencing Benchmarking with Donor Specific Assemblies
• Recalibration for RNA editing sites: High precision Detection of RNA Editing Sites
• Polygenic risk score performance and variant filtering: Breast cancer polygenic risk score performance varies by socioeconomic status
• Machine learning frameworks for medical prediction (context on model validation): Wave aware mortality prediction in COVID 19
• Multi type PTM site prediction (example of annotation based classification): UniPTMs

Related Articles

• Viral Consensus Sequence Quality
• Viral Genome Coverage
• Phylogenetic Root Choice
• Phylogenetic Bootstrap Interpretation
• Molecular Clock Sampling Design