# Hybrid Cloud Variant Calling: How to Choose

Hybrid cloud variant calling means running one variant-calling pipeline across two compute environments, usually an on-premise HPC cluster and a public cloud such as AWS, Azure, or Google Cloud, so that each stage executes where it is cheapest, fastest, or most compliant. The rule of thumb is simple: run per-sample, embarrassingly parallel jobs (alignment, per-sample calling) wherever elastic capacity is cheapest, keep joint genotyping, data staging, and any stage touching the master dataset on the system that already holds the data, and only split the pipeline if your workflow manager can orchestrate both sides without manual file shuffling. If you cannot express the split in code, you do not have a hybrid pipeline, you have two pipelines glued together by a person with an S3 bucket and hope.

This article walks through the four main architecture options, compares them on the criteria that actually decide real deployments, works through three concrete scenarios, and lists the mistakes that most teams make on their first hybrid attempt.

## What Hybrid Cloud Variant Calling Actually Means

![Four-step flow: per-sample alignment and calling in the cloud, joint genotyping near the data, and one workflow manager coordinating both sides.](/article-images/illustrations/hybrid-cloud-variant-calling-hybrid-split-rule.webp)

Three terms carry the weight here, and each is worth defining before choosing anything.

**Variant calling** is the process of identifying genomic differences between a sample and a reference genome. Small variants include single nucleotide variants (SNVs, single-base substitutions) and indels (short insertions and deletions). Larger events include structural variants (SVs, such as deletions, duplications, and inversions spanning 50 base pairs to megabases) and copy number variants (CNVs). A variant-calling pipeline typically chains read alignment, per-sample calling, joint genotyping across a cohort, quality control, and variant annotation, often totaling dozens of heterogeneous software tools.

**Cloud bioinformatics** is the execution of these pipelines on rented, on-demand compute and storage rather than owned hardware. Public cloud providers offer object storage, ephemeral virtual machines, container orchestration, and serverless functions (event-triggered code that runs without managing servers), which map naturally onto genomics workloads that arrive in bursts.

**Hybrid cloud**, in the infrastructure sense used throughout this article, is the deliberate distribution of one workflow across an on-premise environment (typically a Slurm-scheduled HPC cluster funded by a institutional allocation) and one or more public cloud environments. Mulone and colleagues describe exactly this pattern for variant calling, noting that HPC is not always the most suitable environment for an entire pipeline when it involves many heterogeneous tools, and that the ability to execute parts of the pipeline in different environments can lead to higher performance and cheaper executions [1].

One disambiguation matters before going further. In the variant-calling literature, "hybrid" also refers to **hybrid sequencing pipelines** that combine short-read and long-read data from the same sample. That is a different decision entirely, and this article covers it in its own section below, because the two meanings of "hybrid" collide constantly in search results and project meetings. The rest of this article uses "hybrid" in the infrastructure sense unless stated otherwise.

## The Main Hybrid Cloud Variant Calling Options

Four architecture patterns cover nearly every real deployment. Each is described on its own terms, with its pros and cons.

### Option 1: All-Cloud Execution with Programmatic Orchestration

Everything runs in the cloud, but the "hybrid" character comes from mixing cloud primitives: serverless functions for lightweight coordination and small tasks, container orchestration for heavy compute, and object storage as the single source of truth.

The clearest proof of this pattern at scale is Celeste, the cloud architecture behind the All of Us Research Program, one of the world's largest sequencing efforts targeting genetic data for over one million individuals. Celeste is built on serverless technology and container orchestration within a single Amazon Web Services region, and has routinely processed fluctuating workloads of up to 9,000 whole-genome sequencing (WGS) samples per month. In 12 months of production, roughly 200 million serverless functions and over 20 million messages coordinated 1.8 million bioinformatics, quality control, and clinical reporting jobs [2]. The design lesson is that cloud-native orchestration can absorb wildly fluctuating workloads without capacity planning, which is precisely the property a hybrid architecture borrows when it bursts to the cloud.

The economics of all-cloud variant calling were demonstrated early by the 1000 Genomes Project Phase 3 reanalysis. Shringarpure and colleagues deployed the Real Time Genomics population caller on the Amazon cloud through the DNAnexus platform, jointly calling 68.3 million variants across 2,535 samples in 5 days at a compute cost of $7.33 per sample, for a total of $18,590 for completed jobs and $21,805 for all submitted jobs [3]. Their cost analysis suggested that, given near-linear scalability, cloud computing is a cheap and efficient platform for even larger sequencing studies [3].

The same all-cloud pattern now spans providers and contexts. The CIMAC-CIDC cancer immunotherapy network implemented modular WES and RNA-Seq workflows in Snakemake and Docker on Google Cloud Platform, benchmarking improved reproducibility, precision, and recall across validated truth sets [4]. Microsoft-documented work demonstrates Cromwell on Azure handling PacBio HiFi long-read variant calling merged with clinical FHIR data in a single scalable pipeline [5]. ArCH, an end-to-end cloud-based pipeline for clonal hematopoiesis variant calling, accepts varied inputs with customizable parameters across multiple sequencing technologies and datasets [6]. Even data-model-level optimizations exist, with Aizad and colleagues representing Variant Call Format (VCF) files as directed acyclic graphs specifically to enable more efficient and cost-effective cloud analysis [7].

**Pros.** Effectively unlimited elasticity, no capital expenditure, billing maps directly to samples processed, and every result is born next to cheap long-term object storage. Reproducibility is strong because containers plus a workflow definition pin the entire environment.

**Cons.** Egress and cross-region transfer costs punish careless architecture, the bill scales with samples so a runaway joint-calling stage can hurt, and teams inherit the operational learning curve of cloud identity, networking, and cost controls. Data that must stay on institutional storage for policy reasons requires an explicit governance design.

### Option 2: HPC-Primary with Cloud Burst

The cluster remains the home system. Steady-state work runs on the HPC allocation you already pay for, and the cloud absorbs overflow when the queue is full or a deadline compresses the schedule.

This is the pattern the StreamFlow work was built to test. Mulone and colleagues designed and optimized a state-of-the-art variant-calling hybrid workflow on the StreamFlow [workflow management](/knowledge/bioinformatics/snakemake-vs-nextflow-workflow-architectures) system (WfMS), comparing it with Snakemake, an established WfMS targeting HPC facilities. They observed comparable performance when both ran on a single environment, and satisfactory improvements with the hybrid cloud-HPC configuration [1]. The practical takeaway is that the hybrid gain is real but conditional: it appears when the workload actually exceeds local capacity, not as an automatic speedup.

**Pros.** You extract full value from hardware you have already funded, cloud spend becomes a variable and interruptible line item, and sensitive data can stay on the cluster by default with only the parallelizable stages bursting out.

**Cons.** The workflow must genuinely support multi-environment execution. StreamFlow and a small number of other systems handle this natively, while many popular WfMS options require awkward workarounds or per-stage container shuffling. Network staging between the cluster and the cloud can silently dominate runtime if large intermediate files (BAMs, gVCFs) cross the boundary repeatedly. Staff need competence in both Slurm-era tooling and cloud IAM (identity and access management).

### Option 3: Cloud-Primary with On-Premise Stages for Specialized Work

The inverse of Option 2. Production runs in the cloud, but specific stages return to on-premise hardware. Common reasons: a licensed tool restricted to institutional machines, a GPU cluster reserved for deep-learning callers, or an instrument-adjacent server that must see raw data first. The same principle from the StreamFlow study applies in reverse, that executing parts of a pipeline in different environments can improve performance and cost, provided the WfMS coordinates the split rather than a human with rsync [1].

**Pros.** Keeps cloud convenience as the default while honoring license, hardware, or data-placement constraints for narrow stages.

**Cons.** Every crossing of the boundary is a chance for a version mismatch between environments. The on-premise stage becomes a capacity bottleneck that the cloud's elasticity cannot fix, and debugging a failure that spans two environments is genuinely harder than debugging in either one.

### Option 4: Managed Genomics Platforms

A commercial or platform layer (DNAnexus at the time of the 1000 Genomes work [3], or similar managed services) owns the orchestration, autoscaling, and monitoring, and you submit workflows through its interface. The hybrid aspect is that these platforms can often run jobs both in their own cloud tenancy and attached on-premise resources.

**Pros.** The fastest route to production for a team without dedicated infrastructure engineers, with audited access controls that clinical groups often need.

**Cons.** Platform fees sit on top of compute costs, portability suffers if workflows pick up platform-specific features, and the 2015 1000 Genomes cost benchmark of $7.33 per sample reflects that era's prices rather than a current quote [3]. Evaluate current pricing directly with vendors before budgeting.

## Side-by-Side Comparison

| Dimension | All-cloud burst (Option 1) | HPC-primary + cloud burst (Option 2) | Cloud-primary + on-prem stage (Option 3) | Managed platform (Option 4) |
|--|--|--|--|--|
| Where steady-state compute runs | Public cloud | On-premise HPC | Public cloud | Platform tenancy or mixed |
| Cost shape | Fully variable, per sample | Mostly fixed allocation, variable overflow | Mostly variable, fixed island for one stage | Variable compute plus platform fees |
| Elasticity ceiling | Highest (thousands of samples per month demonstrated at All of Us scale [2]) | Bounded by local cluster plus burst capacity | Bounded by the on-premise stage | High, vendor-dependent |
| Workflow manager demand | Low, mature cloud-native WfMS work well | Highest, needs true multi-environment WfMS such as StreamFlow [1] | High, same requirement as Option 2 | Lowest, platform abstracts it |
| Reproducibility mechanism | Containers plus pinned workflow in one environment [4] | Must pin environments on both sides, harder | Must pin environments on both sides | Platform-managed, good for audit [8] |
| Data gravity burden | Low if all data lives in cloud object storage | High, staging costs at the boundary | Highest, repeated crossings | Low if data is ingested once |
| Best fit | Population-scale projects, fluctuating workloads | Labs with funded clusters and bursty overflow | License- or hardware-constricted stages | Clinical or multi-site teams needing audit trails |
| Main risk | Egress and cost-control discipline | Splitting without a capable WfMS | Version drift between environments | Lock-in and fee stacking |

## Comparing Hybrid Cloud Variant Calling Options on the Criteria That Decide Real Choices

Marketing pages compare features. Deployments live or die on five criteria.

### Data Gravity and Transfer Cost

Data gravity is the tendency of large datasets to attract computation, because moving petabyte-scale BAM and gVCF files costs both money and wall-clock time. Every hybrid design is secretly a data-placement decision. A rule grounded in the VCF-as-DAG work: represent and store variants so that downstream cloud computation can consume compact graph structures rather than shuffling monolithic VCFs across the network, which is the explicit motivation Aizad and colleagues give for their representation [7]. In practice this means per-sample outputs should be written directly to object storage reachable by both environments, and joint stages should run where the cohort's data already lives. If your on-premise cluster holds the CRAMs and the cloud holds the callers, you have designed an expensive file-transfer service with a genomics side effect.

### Cost Model and Budget Shape

Cloud cost is usage-based, HPC cost is allocation-based, and the right hybrid option depends on which shape your funding tolerates. The best historical data point comes from the 2,535-genome reanalysis: parallel execution on the Amazon cloud processed the full cohort within 5 days at $7.33 per sample, with cost dependence analysis suggesting near-linear scalability for larger studies [3]. Multiply your sample count by a current per-sample figure from your own pilot run, not from any published number, because compute prices and pipeline efficiency have both moved since 2015. Celeste's experience adds the counterpoint that serverless-heavy architectures trade per-second billing for enormous coordination overhead, roughly 200 million function invocations in a single production year [2], which is efficient at population scale but overkill for a 50-sample study. A useful budgeting heuristic: if your compute need is spiky and unpredictable, variable cloud spend fits. If it is steady and you already own idle cluster capacity, bursting should be the exception, not the plan.

### Reproducibility and Benchmarking

Reproducibility in variant calling has a specific technical meaning: identical inputs produce identical calls regardless of the hardware, operating system, or library versions underneath. Krishnan and colleagues showed why this matters for hybrid designs, since benchmarking methods such as hap.py, vcfeval, and vcflib depend on the underlying hardware and software libraries, making analytical assessment impossible to repeat reliably when dependencies change, and their answer was a scalable, reproducible cloud-based benchmarking workflow independent of the laboratory, technician, and compute hardware [8]. Translate that into hybrid practice directly: benchmark your pipeline in the exact environment mix where it will run, using Genome in a Bottle (GIAB) reference samples, and re-benchmark whenever any side of the hybrid changes. The CIMAC-CIDC team's benchmarking across validated truth sets after migrating their pipelines to GCP illustrates the discipline of revalidating after any environment change [4].

### Workflow Manager Portability

![Comparison of StreamFlow, Snakemake and Cromwell on hybrid support, single-environment performance and where each fits.](/article-images/illustrations/hybrid-cloud-variant-calling-wfms-portability.webp)

This criterion eliminates more hybrid projects at the design stage than any other. The WfMS must express the split declaratively. StreamFlow was designed for exactly this and delivered satisfactory improvements in the hybrid cloud-HPC configuration, while Snakemake performed comparably only within a single environment [1]. Snakemake remains a reasonable choice when the same workflow definition targets either HPC or cloud, as snpArcher demonstrates, being explicitly designed for compatibility with both HPC clusters and cloud environments for population-scale variant calling in nonmodel organisms [9]. The practical test before committing: can your WfMS describe a task that runs on a remote cloud executor, reads inputs from and writes outputs to object storage, and appears in the same provenance log as your cluster tasks, with no human copying files? If not, choose a different option or a different WfMS.

### Security, Clinical Use, and Multi-Site Harmonization

Clinical and multi-site workloads change the calculus. The benchmarking framework for lab-developed tests exists precisely because clinical assays need continual performance assessment across their reportable range, executed identically regardless of who runs it or where [8]. Multi-site consortia face a harmonization problem: the CIMAC-CIDC network rebuilt its pipelines around open-source tools and containers specifically to ensure continuity and reliability of analyses across sites during an organizational migration [4]. ArCH shows the same pattern in somatic variant calling for clonal hematopoiesis, where a customizable, end-to-end cloud pipeline was built to be adaptable across sequencing technologies and research questions [6]. For clinical teams, the managed platform option often wins not on technical merit but on audit trails and access controls that regulators and collaborators recognize. For research consortia, containerized workflows with pinned versions are the mechanism that keeps a variant call the same variant call at every participating center.

A subtler consistency issue comes from association studies. When cases and controls are processed by different pipelines, test-statistic inflation follows. Kim and colleagues found that inflated distributions of test statistics in rare-variant analyses decreased substantially when the same variant caller and filtering pipelines were employed, while differences in library prep kits and sequencers did not drive inflation, and joint versus separate calling did not contribute [10]. The hybrid-cloud lesson is direct: whatever you split across environments, do not split the caller and filtering chain, because pipeline heterogeneity is a demonstrated source of false-positive findings.

## Worked Scenarios

### Scenario 1: The University Lab with a Fixed HPC Allocation

A group of eight has a funded Slurm cluster, a three-year grant, and a fluctuating sample inflow that occasionally exceeds the queue. Their data is mostly small (exome panels and target enrichment). The right choice is Option 2. They keep steady-state processing on the cluster, wrap every task in a container, and write their workflow so that the same tasks can execute on a cloud executor during deadline crunches. Before committing, they should honestly assess whether their WfMS supports remote execution, given that comparable performance across single environments and real gains only in hybrid configuration is exactly what the StreamFlow comparison demonstrated. If their WfMS cannot do this, the honest fallback is Option 4 for overflow only, paying platform fees for the burst fraction rather than rebuilding their stack.

### Scenario 2: The Population-Scale Sequencing Program

A consortium is scaling toward tens of thousands of WGS samples with wildly fluctuating monthly inflow. Their data can live in a single cloud region under an appropriate data-governance agreement. The evidence supports Option 1. Celeste's architecture handled fluctuating workloads of up to 9,000 WGS samples monthly within one AWS region, coordinating 1.8 million jobs over a production year [2], and the 1000 Genomes reanalysis established the per-sample economics of cloud joint calling years earlier [3]. The design work they should budget for is not compute but coordination: serverless orchestration, idempotent task retry, and cost controls. They should also pin their reference and benchmarking strategy early, because reference choice itself affects calls, as the T2T-CHM13 work demonstrated by showing that GRCh38's unresolved bases introduce reference biases that cause missed variants and false positives [11].

### Scenario 3: The Clinical Lab Developing an Assay

A hospital laboratory is validating a germline panel as a lab-developed test and must demonstrate reproducible analytical performance across its reportable range. Their institutional policy restricts where patient data may rest. Option 4, or a tightly scoped Option 3, fits. They should adopt a cloud-based benchmarking workflow independent of the executing technician and hardware, run continuously against GIAB samples, exactly as Krishnan and colleagues describe [8]. If any stage must remain on-premise, they should version-pin that stage with the same rigor as the cloud stages, because the demonstrated inflation from pipeline heterogeneity in association settings [10] is a proxy for the consistency risk that clinical validation cannot tolerate. If structural variants need manual review, a cloud-based curation framework such as SV-plaudit, which let nine researchers evaluate 1,350 SVs each in a single hour through a shared web application, shows how review itself can be centralized and standardized [12].

## A Note on Hybrid Sequencing Data in Hybrid Cloud Pipelines

The word "hybrid" also describes pipelines that combine short-read and [long-read sequencing](/knowledge/bioinformatics/long-read-sequencing-technologies-pacbio-and-oxford-nanopore) from the same sample, and this meaning intersects directly with infrastructure choices because hybrid data pipelines multiply data volumes and tool heterogeneity, which is exactly what strains a compute split.

The biology case is strong. Sentieon's DNAscope Hybrid pipeline integrated low-depth ONT reads with Illumina short reads and achieved F1 scores of 0.9992 for SNPs and 0.9979 for indels, outperforming alternative pipelines, while DNAscope LongRead reduced SNP errors by roughly 50% relative to Clair3 across five reference samples [13]. The same group's broader hybrid benchmark across HG002 to HG004 evaluated SNP, indel, SV, and CNV detection from Illumina and PacBio data at depths from 5x to 30x against DeepVariant [14]. [Preprocessing](/blog/data-analysis/what-is-data-preprocessing) matters too: Ratatosk corrects long reads with short-read data, reducing long-read error rates six-fold on average to a median of 0.22%, with nearly 99% accurate SNP calls in corrected reads [15]. HELLO takes a different route, using a Mixture of Experts ensemble of deep neural networks in which each network processes a different data type to call small variants jointly from short and long reads [16].

The infrastructure consequence is that hybrid sequencing inputs widen the gap between pipeline stages, since long-read alignment, short-read alignment, error correction, and ensemble calling have different hardware profiles. If your biological question demands hybrid sequencing data, your compute architecture almost certainly demands a workflow manager that can place those heterogeneous stages appropriately, which circles back to Option 2 or Option 3 with a capable WfMS.

## Common Mistakes and Limitations

**Splitting the pipeline without a WfMS that supports it.** The most common failure is assuming any workflow tool can orchestrate across environments. StreamFlow was purpose-built for the hybrid cloud-HPC configuration and delivered its gains there, while an established HPC-targeting WfMS performed comparably only on single environments [1]. Test the split with a two-task toy workflow before betting a project on it.

**Benchmarking in one environment and producing results in another.** Benchmarking methods depend on hardware, operating system, and software libraries, which makes analytical assessments unreliable to repeat when dependencies change [8]. Validate in the production environment mix, and revalidate after every environment change, container update, or instance-family switch.

**Ignoring data gravity until the bill arrives.** Design the placement of BAMs, gVCFs, and joint-call inputs before writing workflows, not after the first month of transfer charges. Compact representations such as the VCF-as-DAG approach exist specifically to reduce the cost of moving variant data for cloud analysis [7].

**Changing the caller or filter chain between sample groups.** If cases are processed on the cluster and controls on the cloud with a different filtering chain, association results can inflate. The same caller and filtering pipelines materially reduced test-statistic inflation in the public-controls study, and pipeline accessibility was central to that finding [10].

**Assuming a caller tuned for one data type transfers to another.** ONT's error profile, particularly indels in homopolymer regions, requires platform-aware methods, which is why purpose-built pipelines reduced SNP errors by roughly half relative to a general long-read caller [13]. A pipeline validated on Illumina data is not validated on ONT data, and a pipeline validated on GRCh38 is not validated on T2T-CHM13, where reference-dependent bias changes what is callable [11].

**Treating [structural variant](/blog/guides/structural-variant) review as an afterthought.** SV callers produce candidates that benefit from human review, and uncoordinated review does not scale. Centralized cloud curation frameworks exist for exactly this stage and have demonstrated throughput of thousands of SVs per reviewer-hour in a structured setting [12].

**Assuming hybrid always wins.** The StreamFlow study itself observed comparable performance on single environments, with hybrid gains appearing only in the mixed configuration. If your cluster is never saturated and your deadline pressure is mild, a pure HPC or pure cloud deployment may beat a hybrid one on simplicity and total cost.

**Underestimating the operational surface.** A hybrid pipeline inherits two identity systems, two monitoring stacks, two failure modes, and two sets of quotas. Teams that budget engineering time only for the biology consistently under-resource the glue.

## Frequently Asked Questions

### What is hybrid cloud variant calling?

It is the practice of running a single variant-calling pipeline across both an on-premise environment, usually an HPC cluster, and a public cloud, assigning each pipeline stage to the environment where it runs cheapest or fastest. A purpose-built workflow management system coordinates the split, since manual file transfers between environments do not scale and undermine reproducibility.

### Is hybrid cloud variant calling cheaper than running everything on our own HPC?

It depends on your workload shape and existing capacity. Cloud processing of the 2,535-genome 1000 Genomes Phase 3 cohort cost $7.33 per sample in compute and finished in 5 days, with near-linear scalability suggested for larger studies [3]. If your cluster sits idle, adding cloud spend raises cost. If your cluster is saturated and deadlines compress, bursting overflow to the cloud is typically the economical choice, and measured improvements in hybrid cloud-HPC configurations support this in practice.

### Which workflow managers support hybrid cloud variant calling?

StreamFlow was designed specifically for hybrid cloud-HPC variant calling and showed satisfactory improvements in that configuration [1]. Snakemake supports workflows that target both HPC clusters and cloud environments, as demonstrated by the snpArcher population genomics pipeline [9]. Other WfMS options, including Cromwell and cloud-native serverless architectures, support cloud execution but vary in how natively they express a split between environments [5][2].

### How do I validate a variant calling pipeline before trusting cloud results?

Benchmark against Genome in a Bottle reference samples using standard tools such as hap.py or vcfeval, and run the benchmark in the exact environment mix where production will run, because benchmarking performance depends on the underlying hardware and software libraries [8]. Multi-site and clinical teams should favor containerized, cloud-based benchmarking workflows that produce identical assessments regardless of who executes them or where [4].

### Does AWS work for genomics workloads at scale?

Yes, and the evidence is at population scale. The All of Us program's Celeste architecture processed fluctuating workloads of up to 9,000 WGS samples per month within a single AWS region, coordinating roughly 200 million serverless functions and 1.8 million jobs in a production year [2], and the 1000 Genomes reanalysis ran joint variant calling on the Amazon cloud through DNAnexus years earlier [3]. The design burden shifts from provisioning hardware to orchestrating serverless coordination and controlling data-transfer costs.

### When should we not use a hybrid cloud setup?

Skip the hybrid split when your existing cluster never saturates, your data governance forbids cloud placement of the master dataset, or your workflow manager cannot express remote execution without manual intervention. In the measured comparison, a hybrid configuration added value only over a single-environment baseline when the workload actually required it, so simplicity wins for small, steady, unconstrained workloads.

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [
    {
      "@type": "Question",
      "name": "What is hybrid cloud variant calling?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "It is the practice of running a single variant-calling pipeline across both an on-premise environment, usually an HPC cluster, and a public cloud, assigning each pipeline stage to the environment where it runs cheapest or fastest. A purpose-built workflow management system coordinates the split, since manual file transfers between environments do not scale and undermine reproducibility."
      }
    },
    {
      "@type": "Question",
      "name": "Is hybrid cloud variant calling cheaper than running everything on our own HPC?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "It depends on your workload shape and existing capacity. Cloud processing of the 2,535-genome 1000 Genomes Phase 3 cohort cost $7.33 per sample in compute and finished in 5 days. If your cluster sits idle, adding cloud spend raises cost. If your cluster is saturated and deadlines compress, bursting overflow to the cloud is typically the economical choice."
      }
    },
    {
      "@type": "Question",
      "name": "Which workflow managers support hybrid cloud variant calling?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "StreamFlow was designed specifically for hybrid cloud-HPC variant calling and showed satisfactory improvements in that configuration. Snakemake supports workflows that target both HPC clusters and cloud environments, as demonstrated by the snpArcher pipeline. Other options, including Cromwell and serverless cloud-native architectures, support cloud execution but vary in how natively they express a split between environments."
      }
    },
    {
      "@type": "Question",
      "name": "How do I validate a variant calling pipeline before trusting cloud results?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Benchmark against Genome in a Bottle reference samples using standard tools such as hap.py or vcfeval, and run the benchmark in the exact environment mix where production will run, because benchmarking performance depends on the underlying hardware and software libraries. Multi-site and clinical teams should favor containerized, cloud-based benchmarking workflows that produce identical assessments regardless of who executes them or where."
      }
    },
    {
      "@type": "Question",
      "name": "Does AWS work for genomics workloads at scale?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Yes, and the evidence is at population scale. The All of Us program's Celeste architecture processed fluctuating workloads of up to 9,000 WGS samples per month within a single AWS region, coordinating roughly 200 million serverless functions and 1.8 million jobs in a production year, and the 1000 Genomes reanalysis ran joint variant calling on the Amazon cloud through DNAnexus years earlier."
      }
    },
    {
      "@type": "Question",
      "name": "When should we not use a hybrid cloud setup?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Skip the hybrid split when your existing cluster never saturates, your data governance forbids cloud placement of the master dataset, or your workflow manager cannot express remote execution without manual intervention. Measured comparisons show a hybrid configuration added value only when the workload actually required it, so simplicity wins for small, steady, unconstrained workloads."
      }
    }
  ]
}
</script>

## Sources

1. [Porting the Variant Calling Pipeline for NGS data in cloud-HPC environment](https://doi.org/10.1109/COMPSAC57700.2023.00288)
2. [Celeste: A cloud-based genomics infrastructure with variant-calling pipeline suited for population-scale sequencing projects](https://pubmed.ncbi.nlm.nih.gov/40343041/)
3. [Inexpensive and Highly Reproducible Cloud-Based Variant Calling of 2,535 Human Genomes](https://pubmed.ncbi.nlm.nih.gov/26110529/)
4. [Modular and cloud-based bioinformatics pipelines for high-confidence biomarker detection in cancer immunotherapy clinical trials](https://pubmed.ncbi.nlm.nih.gov/40857351/)
5. [A cloud-based pipeline for analysis of FHIR and long-read data.](https://pubmed.ncbi.nlm.nih.gov/36726729/)
6. [ArCH: improving the performance of clonal hematopoiesis variant calling and interpretation](https://pubmed.ncbi.nlm.nih.gov/38485690/)
7. [Representing Variant Calling Format as Directed Acyclic Graphs to Enable the Use of Cloud Computing for Efficient and Cost Effective Genome Analysis](https://doi.org/10.1109/CCGRID.2017.116)
8. [Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays](https://pubmed.ncbi.nlm.nih.gov/33627090/)
9. [A Fast, Reproducible, High-throughput Variant Calling Workflow for Population Genomics](https://pubmed.ncbi.nlm.nih.gov/38069903/)
10. [Inflated expectations: Rare-variant association analysis using public controls.](https://pubmed.ncbi.nlm.nih.gov/36696392/)
11. [Telomere-to-telomere CHM13 reference reveals missing truth variants and improves deep learning-based variant calling in long-read sequencing data](https://pubmed.ncbi.nlm.nih.gov/42845716/)
12. [SV-plaudit: A cloud-based framework for manually curating thousands of structural variants.](https://pubmed.ncbi.nlm.nih.gov/29860504/)
13. [Accelerated and High-Accuracy Variant Calling on Oxford Nanopore Technologies Sequencing Data with the Sentieon DNAscope LongRead and Hybrid Pipelines](https://doi.org/10.1101/2025.11.14.688541)
14. [Accelerated, Accurate, Hybrid Short and Long Reads Alignment and Variant Calling](https://doi.org/10.1101/2025.04.15.648987)
15. [Ratatosk: hybrid error correction of long reads enables accurate variant calling and assembly](https://pubmed.ncbi.nlm.nih.gov/33419473/)
16. [HELLO: A hybrid variant calling approach](https://doi.org/10.1101/2020.03.23.004473)