# Snakemake on Google Cloud Life Sciences By the end of this guide you will be able to launch a Snakemake pipeline on Google Cloud so that every rule runs as a job on its own virtual machine, reads its input from Cloud Storage, and writes results back to a bucket you control, all coordinated from your laptop. You will also understand what happened to the original Google Cloud Life Sciences backend, which errors cause most failed first runs, and how Snakemake compares with Nextflow for cloud execution. Have on hand a Google Cloud account with billing enabled, the gcloud CLI installed, Miniforge or Miniconda, and one or two small FASTQ files for testing. The walkthrough takes roughly 30 minutes the first time. A bioinformatics workflow manager is software that chains analysis steps into a pipeline, records which files each step consumes and produces, and reruns only the steps a change actually affects. Snakemake, a Python-based workflow manager, and Nextflow are the two options you will encounter most often in bioinformatics. One naming note before any commands. Running Snakemake on Google Cloud Life Sciences was once a documented, first-class path. Google Cloud Life Sciences was a GCP product, an API that ran each pipeline step on a Compute Engine VM and moved files to and from Cloud Storage automatically. Google deprecated that API and retired it in 2024, and the same job is now done by Google Batch. Snakemake runs on Google Cloud today through a Google Batch executor plugin, and this guide uses that current path throughout. ## What happened to the Google Cloud Life Sciences backend Snakemake 5.9 through 7.x shipped a native executor for the Cloud Life Sciences API, invoked like this: ```bash # Historical syntax, Snakemake 7.x, no longer functional snakemake --google-lifesciences \ --google-lifesciences-regions us-central1 \ --google-lifesciences-machine-type n1-standard-8 \ --use-singularity \ --default-remote-prefix gs://my-bucket/workflow ``` Under that backend, each rule became a Life Sciences pipeline run. The API localized inputs from Cloud Storage to the VM, ran the rule command inside a Singularity container (containers were required for every rule), and copied outputs back to the bucket. Setup meant enabling `lifesciences.googleapis.com`, creating a service account with `roles/lifesciences.workflowsRunner` plus service account user and storage roles, and writing a boto credential file at `~/.config/gcloud/boto` that pointed at the service account key so the old `gs://` remote could authenticate. Google deprecated the Cloud Life Sciences API and shut it down in 2024, pointing users to Google Batch. Snakemake 8 then reorganized all executors and storage backends into plugins and did not carry the Life Sciences executor forward, because there was no service left to talk to. If you follow a 2020 tutorial today, the command fails at argument parsing on current Snakemake, or at the API call on old Snakemake. The replacement keeps the architecture and changes the plumbing. Rules still become one VM job each, Cloud Storage still acts as the shared file system, but jobs go through Google Batch and file access goes through Snakemake's storage plugin system. ## Why run Snakemake on Google Cloud at all Three reasons dominate in practice. First, elastic scale. A whole-genome alignment or variant calling run can burst to dozens of VMs for a few hours and cost nothing when idle. For small and mid-market life sciences organizations without compute clusters, this was the core appeal in early adoption studies. A mixed-method investigation of four small biotech and pharmaceutical companies found cloud computing attractive, with documented benefits and drawbacks and clear planning and implementation stages [1]. The same study, which combined action research, direct observation, and review of other case studies, closed with detailed recommendations for organizations planning cloud adoption [2]. Second, provider fit. A comparative analysis of AWS, Azure, and Google Cloud for healthcare and life sciences ventures concluded that GCP stands out on breadth of services, global infrastructure, security measures, and healthcare case studies [3]. Third, the Snakemake model itself. You describe files, not schedulers. Wildcards map inputs to outputs, conda environments or containers pin software, and Snakemake reruns only what changed. Scalable computation for life sciences is not new, workshop special issues on emerging computational methods for the life sciences go back well over a decade [4], and the series continued with parallel and distributed analysis work [5]. Snakemake on managed cloud infrastructure is the current expression of that line of work. ## How the pieces fit together A cloud run has five moving parts. Knowing which part owns which failure saves hours of debugging. 1. The head node is wherever you type `snakemake`. It builds the DAG, submits jobs, and tracks status. It needs almost no compute. 2. Google Batch is the managed job service. It takes each Snakemake job, provisions a Compute Engine VM for it, runs the command, and reports status. 3. Cloud Storage is the shared file system. Every input, output, and log file lives at a `gs://` path. 4. IAM decides who can submit jobs and which identity the job VMs use to read and write the bucket. 5. Cloud Logging holds the stdout and stderr of every Batch task, which is where you look when a job dies. ## Step by step: Snakemake on Google Cloud Life Sciences workloads Today that means the Google Batch executor. The eight steps below take you from an empty project to collected results. ### 1. Prepare the project and authenticate ```bash gcloud auth login gcloud projects create snakemake-demo-2026 gcloud config set project snakemake-demo-2026 gcloud billing projects link snakemake-demo-2026 \ --billing-account=0X0X0X-0X0X0X-0X0X0X ``` Replace the billing account ID with yours, which you can find in the Cloud Console under Billing. Without a billing link, every API call after this step fails. ### 2. Enable the required APIs ```bash gcloud services enable \ batch.googleapis.com \ compute.googleapis.com \ storage.googleapis.com \ iam.googleapis.com \ cloudresourcemanager.googleapis.com ``` Expected output ends with a line like: ``` Operation "operations/acf.p2-1234567890-abcdef" finished successfully. ``` Old tutorials tell you to enable `lifesciences.googleapis.com`. That API is retired. Do not chase it. ### 3. Create the bucket and the service identities Create one bucket in one region and keep everything in that region: ```bash gcloud storage buckets create gs://snakemake-demo-2026-run \ --location=us-central1 \ --uniform-bucket-level-access ``` Now create two identities. The runner service account is what your laptop impersonates when it submits jobs and reads the bucket. The Compute Engine default service account is what the job VMs run as. ```bash gcloud iam service-accounts create snakemake-runner \ --display-name="Snakemake runner" ``` Grant the runner the roles it needs to submit Batch jobs and use the bucket: ```bash PROJECT_ID=snakemake-demo-2026 RUNNER=snakemake-runner@${PROJECT_ID}.iam.gserviceaccount.com gcloud projects add-iam-policy-binding ${PROJECT_ID} \ --member="serviceAccount:${RUNNER}" \ --role="roles/batch.jobsAdmin" gcloud projects add-iam-policy-binding ${PROJECT_ID} \ --member="serviceAccount:${RUNNER}" \ --role="roles/iam.serviceAccountUser" gcloud projects add-iam-policy-binding ${PROJECT_ID} \ --member="serviceAccount:${RUNNER}" \ --role="roles/storage.objectAdmin" ``` Then wire up the VM side. Batch needs the Compute Engine service agent to manage instances, and the VMs need bucket access: ```bash PROJECT_NUMBER=$(gcloud projects describe ${PROJECT_ID} \ --format="value(projectNumber)") DEFAULT_SA="${PROJECT_NUMBER}-compute@developer.gserviceaccount.com" AGENT="service-${PROJECT_NUMBER}@compute-system.iam.gserviceaccount.com" gcloud projects add-iam-policy-binding ${PROJECT_ID} \ --member="serviceAccount:${DEFAULT_SA}" \ --role="roles/storage.objectAdmin" gcloud projects add-iam-policy-binding ${PROJECT_ID} \ --member="serviceAccount:${AGENT}" \ --role="roles/compute.instanceAdmin.v1" gcloud projects add-iam-policy-binding ${PROJECT_ID} \ --member="serviceAccount:${AGENT}" \ --role="roles/iam.serviceAccountUser" ``` Finally, create a key so the Snakemake process on your head node can act as the runner: ```bash gcloud iam service-accounts keys create snakemake-runner-key.json \ --iam-account=${RUNNER} export GOOGLE_APPLICATION_CREDENTIALS="$(pwd)/snakemake-runner-key.json" ``` Treat that key file like a password. Delete it when the project is done with `gcloud iam service-accounts keys delete`. Keyless authentication with `gcloud auth application-default login` also works for the storage plugin and is the better habit on a personal machine. ### 4. Install Snakemake and the Google Cloud plugins ```bash mamba create -n snakemake -c conda-forge -c bioconda "snakemake>=8" mamba activate snakemake pip install snakemake-executor-plugin-googlebatch snakemake-storage-plugin-gcs snakemake --version ``` `snakemake, version` should print an 8.x or 9.x number. Confirm the executor plugin is visible: ```bash pip show snakemake-executor-plugin-googlebatch ``` Snakemake 9 also accepts `-software-deployment-method conda apptainer` as the modern spelling of `-use-conda` combined with `-use-apptainer`. Both spellings work on current versions. ### 5. Write a small pipeline Keep the project self-contained, because the executor stages the workflow definition, including the Snakefile, env files, and any scripts, into the storage prefix so each VM can fetch it. The project needs these files: - `Snakefile` - `config.yaml` - `envs/fastp.yaml`, `envs/fastqc.yaml`, `envs/multiqc.yaml` - `data/raw/` (empty locally, filled in the bucket) The demo pipeline trims reads with fastp, runs FastQC on the raw reads, and summarizes everything with MultiQC. ```yaml # config.yaml samples: - A - B ``` ```yaml # envs/fastp.yaml channels: - conda-forge - bioconda dependencies: - fastp=0.23.2 ``` Create `envs/fastqc.yaml` and `envs/multiqc.yaml` the same way, pinning `fastqc=0.11.9` and `multiqc=1.14`. ```python # Snakefile configfile: "config.yaml" SAMPLES = config["samples"] rule all: input: "results/multiqc/multiqc_report.html", expand("results/trimmed/{sample}.trimmed.fastq.gz", sample=SAMPLES) rule fastp: input: "data/raw/{sample}.fastq.gz" output: reads="results/trimmed/{sample}.trimmed.fastq.gz", json="reports/fastp/{sample}.json", html="reports/fastp/{sample}.html" log: "logs/fastp/{sample}.log" threads: 4 resources: mem_mb=4000, disk_mb=20000 conda: "envs/fastp.yaml" shell: "fastp --in1 {input} --out1 {output.reads} " "--json {output.json} --html {output.html} " "--thread {threads} 2> {log}" rule fastqc: input: "data/raw/{sample}.fastq.gz" output: "reports/fastqc/{sample}.fastq_fastqc.zip" log: "logs/fastqc/{sample}.log" threads: 2 resources: mem_mb=2000, disk_mb=20000 conda: "envs/fastqc.yaml" shell: "fastqc --outdir reports/fastqc --threads {threads} {input} 2> {log}" rule multiqc: input: fastqc=expand("reports/fastqc/{sample}.fastq_fastqc.zip", sample=SAMPLES), fastp=expand("reports/fastp/{sample}.json", sample=SAMPLES) output: "results/multiqc/multiqc_report.html" log: "logs/multiqc.log" threads: 1 resources: mem_mb=2000, disk_mb=20000 conda: "envs/multiqc.yaml" shell: "multiqc --outdir results/multiqc {input} 2> {log}" ``` Parameters that matter here: - Wildcards such as `{sample}` let one rule definition cover every sample. The DAG for two samples is six jobs. - `threads`, `mem_mb`, and `disk_mb` are resource hints. The Google Batch executor uses them when it provisions the job VM, so realistic values matter. - The `log:` directive sends stderr to a file that lands in the bucket, which is your first stop when a job fails. - `conda:` pins the exact software stack per rule, which is what makes the run reproducible. ### 6. Upload data and do a dry run Put your FASTQ files in `data/raw/` locally, named `A.fastq.gz` and `B.fastq.gz` to match the config, then copy them to the prefix the run will use: ```bash gcloud storage cp data/raw/*.fastq.gz \ gs://snakemake-demo-2026-run/workflow/data/raw/ ``` The storage prefix maps every relative path in the Snakefile onto the bucket, so `data/raw/A.fastq.gz` becomes `gs://snakemake-demo-2026-run/workflow/data/raw/A.fastq.gz`. Now build the DAG without running anything: ```bash snakemake \ --executor googlebatch \ --default-storage-prefix gs://snakemake-demo-2026-run/workflow \ --use-conda \ --jobs 10 \ --dry-run ``` Expected output, abridged: ``` Building DAG of jobs... Job stats: job count ------- ------- all 1 fastp 2 fastqc 2 multiqc 1 total 6 ``` If the counts look wrong, fix the Snakefile now. A dry run is free. ### 7. Launch the cloud run ```bash snakemake \ --executor googlebatch \ --default-storage-prefix gs://snakemake-demo-2026-run/workflow \ --use-conda \ --jobs 10 \ --retries 2 \ --latency-wait 120 \ --printshellcmds ``` What each parameter does: - `-executor googlebatch` sends every job to Google Batch instead of running it locally. - `-default-storage-prefix` maps relative paths to the bucket. Without it, cloud jobs look for files on a VM disk that has none. - `-use-conda` builds each rule's environment on the job VM. The first run is slow because every VM starts clean and builds its environment before the command runs. Containers remove that cost, as described in the tips section. - `-jobs 10` caps how many jobs run at once. With fastp claiming 4 threads, ten concurrent fastp jobs would request 40 vCPUs, so a small regional vCPU quota becomes the real ceiling. - `-retries 2` resubmits failed jobs automatically, which absorbs transient API and storage errors. - `-latency-wait 120` gives slow object-store listings time to settle before Snakemake declares an output missing. - `-printshellcmds` prints every command, which makes logs and debugging far easier. Expected console output, abridged: ``` Building DAG of jobs... Job stats: job count ------- ------- all 1 fastp 2 fastqc 2 multiqc 1 total 6 Select jobs to execute... [Wed Nov 11 10:12:03 2026] rule fastqc: input: gs://snakemake-demo-2026-run/workflow/data/raw/A.fastq.gz output: gs://snakemake-demo-2026-run/workflow/reports/fastqc/A.fastq_fastqc.zip log: gs://snakemake-demo-2026-run/workflow/logs/fastqc/A.log jobid: 2 reason: Missing output files: gs://snakemake-demo-2026-run/workflow/reports/fastqc/A.fastq_fastqc.zip resources: mem_mb=2000, disk_mb=20000 Submitted job 2 (external jobid: projects/123456789012/locations/us-central1/jobs/snakemake-a1b2c3). ``` While it runs, you can watch the jobs in the Cloud Console under Batch, or from the command line: ```bash gcloud batch jobs list --location=us-central1 ``` A successful run ends like this: ``` 6 of 6 steps (100%) completed Complete log: .snakemake/log/20261111T101201.987654.snakemake.log ``` ### 8. Verify the results and collect them Check that every expected object exists in the bucket: ```bash gcloud storage ls "gs://snakemake-demo-2026-run/workflow/results/**" ``` Expected listing: ``` gs://snakemake-demo-2026-run/workflow/results/multiqc/multiqc_report.html gs://snakemake-demo-2026-run/workflow/results/trimmed/A.trimmed.fastq.gz gs://snakemake-demo-2026-run/workflow/results/trimmed/B.trimmed.fastq.gz ``` Then download the results folder and open the MultiQC report in a browser: ```bash gcloud storage cp -r \ gs://snakemake-demo-2026-run/workflow/results ./results ``` For a browsable record of the whole run, including per-job statistics, generate a Snakemake report from the local metadata: ```bash snakemake --report report.html ``` Run the exact same command a second time. Snakemake checks its metadata and the bucket, finds everything current, and prints `Nothing to be done (all requested files are present and up to date).` That is the correctness check for the resume machinery, and it is the behavior you will rely on when a 500-sample run dies at job 499. Sensible next steps: add more samples to `config.yaml`, add alignment and variant calling rules with larger `mem_mb` values, switch the run to containers, and set a billing alert in the Cloud Console before the first large run. ## Where your files live during a cloud run The storage model is the biggest mental shift from laptop Snakemake. - The head node keeps the workflow definition and the `.snakemake/` directory, which holds job metadata and local logs. This is small and belongs in version control. - The bucket holds inputs, outputs, and rule logs. The `-default-storage-prefix` maps relative path `P` to `gs:///P`. Any path you write with an explicit `gs://` prefix passes through unchanged. - The executor stages the workflow definition into the prefix so each VM can fetch it, which is why env files and scripts must live inside the project folder. - Intermediates you do not want to keep can be wrapped in `temp()`, for example `temp("results/trimmed/{sample}.trimmed.fastq.gz")`, and Snakemake deletes them from the bucket once no downstream job needs them. Keep the bucket and the Batch jobs in the same region. Cross-region reads are slower and you pay for the traffic. When a test project is finished, remove its objects with: ```bash gcloud storage rm -r gs://snakemake-demo-2026-run/workflow ``` ## Sizing jobs and monitoring a cloud run Set resources per rule, and provide a fallback for rules that omit them: ```bash snakemake \ --executor googlebatch \ --default-storage-prefix gs://snakemake-demo-2026-run/workflow \ --use-apptainer \ --default-resources mem_mb=2000 disk_mb=20000 \ --jobs 20 ``` When a job fails, triage in this order: the Snakemake console log, the rule's `log:` file in the bucket, then the Batch task output in Cloud Logging. The Batch console page and `gcloud batch jobs describe JOB_NAME, location=us-central1` show the VM state, exit codes, and timing. | Flag | What it does | Example | |, - |, - |, - | | `-executor googlebatch` | Sends each job to Google Batch | `-executor googlebatch` | | `-default-storage-prefix` | Maps relative paths to a GCS prefix | `-default-storage-prefix gs://my-bucket/workflow` | | `-use-conda` | Builds the rule's conda env on the job VM | `-use-conda` | | `-use-apptainer` | Runs each rule inside its container image | `-use-apptainer` | | `-jobs` | Caps concurrent cloud jobs | `-jobs 20` | | `-default-resources` | Fallback resource hints for rules without any | `-default-resources mem_mb=2000 disk_mb=20000` | | `-retries` | Resubmits failed jobs automatically | `-retries 2` | | `-latency-wait` | Waits for slow object-store listings before declaring files missing | `-latency-wait 120` | | `-printshellcmds` | Prints every executed command | `-printshellcmds` | | `-max-status-checks-per-second` | Throttles polling of the Batch API | `-max-status-checks-per-second 0.5` | ## Practical tips for Snakemake on Google Cloud 1. Pin versions in every conda env file. `fastp` today and `fastp` next year are different programs. 2. Prefer containers for cloud runs. Add a `container:` directive and run with `-use-apptainer`, for example `container: "docker://quay.io/biocontainers/fastqc:0.11.9-hdfd78af_1"`. An image pulls once per VM, while a conda environment is rebuilt for every job. 3. Always pass `-printshellcmds` on cloud runs. The command line in the log is what you compare against the Batch task output when something fails. 4. Test small before scaling. Two samples with `-jobs 2` exercises the entire IAM, storage, and executor path in minutes. 5. Use `config.yaml` for sample lists so the identical Snakefile runs on a laptop and in the cloud. 6. Use `-retries 2` on every cloud run. Transient storage and API errors are normal at scale. 7. Use `-latency-wait 120` with object storage. Files can exist before listings reflect them. 8. Use `-omit-from` or `-until` to exercise one branch of the DAG during development instead of paying for the full run. 9. Set a billing alert in the Cloud Console before the first large run, and delete test buckets when finished. 10. Back up the workflow directory including `.snakemake/`. Losing the metadata forces full reruns. ## Common Mistakes and Limitations **Running retired commands from old tutorials.** `-google-lifesciences` fails on Snakemake 8 and newer with an error saying the executor is unknown and pointing you to a plugin, and no Life Sciences plugin exists because the API is gone. On old Snakemake versions it fails at the API call. The fix is the Google Batch path in this guide. **Missing storage plugin.** With `gs://` paths and no `snakemake-storage-plugin-gcs` installed, Snakemake fails while building the DAG and names the unavailable storage provider. Install both plugins from step 4. **Inputs in the wrong prefix.** The classic failure looks like this: ``` MissingInputException in line 30 of /home/you/workflow/Snakefile: Missing input files for rule fastqc: input: data/raw/A.fastq.gz ``` The file exists, just not at `gs://your-prefix/data/raw/A.fastq.gz`. Upload to the exact prefix you pass to `-default-storage-prefix`. **IAM gaps.** These surface at two distinct points. A permission error during submission means the runner lacks a Batch or service account user role. A 403 inside the task logs in Cloud Logging means the VM service account cannot read or write the bucket. Test the runner credentials with `gcloud storage ls` before launching. **Quota ceilings.** New projects carry small regional vCPU quotas, and a run that requests more fails with messages like `Quota 'CPUS' exceeded. Limit: 12.0 in region us-central1.` Lower `-jobs`, pick a less busy region, or request an increase in the IAM and admin Quotas page. **Killed jobs from undersized memory hints.** A job that exceeds its VM memory dies with exit code 137. Raise `mem_mb` on the rule and let `-retries` absorb the failed attempt. **Impatience with object-store listings.** Snakemake prints `Missing files after 5 seconds.` when an expected output is not visible yet, which happens with cloud storage even when the file landed. Raise `-latency-wait` instead of rerunning. **Conda rebuilds on every VM.** With `-use-conda`, each job VM builds its environment from scratch before running the command. For many short jobs this dominates wall time. Switch to containers with `-use-apptainer` for production runs. **Wildcard typos.** A mismatch between the wildcards used in input and output patterns produces: ``` WildcardError in line 41 of /home/you/workflow/Snakefile: Wildcards in input files of rule multiqc cannot be determined from output files. ``` Check spelling first, then constrain ambiguous patterns with `wildcard_constraints`. **Cross-region data gravity.** A bucket in `europe-west1` with jobs in `us-central1` is slow and generates inter-region traffic. Co-locate bucket, jobs, and region. **Treating the head node as disposable.** Job metadata lives in `.snakemake/` on the machine that launched the run. Delete it and Snakemake no longer knows what it did, so the next invocation reruns completed work. Keep the directory in git or a backup. **Unpinned environments.** An env file listing `fastp` without a version produces different results across months. Pin everything. Limitations to keep in mind. Plugin defaults and flag behavior evolve between Snakemake releases, so confirm against the plugin documentation for the version you install. Automated pipeline output still needs expert biological interpretation before any downstream decision. And for regulated work, validation obligations do not disappear when the pipeline moves to managed services, a point early cloud adopters in the life sciences also learned about security and management overhead. Conventional computer system validation assumed static, on-premises systems and copes poorly with dynamic, scalable, distributed cloud platforms, which is why risk-based approaches to cloud compliance have been proposed for life sciences [6]. ## Nextflow vs Snakemake on Google Cloud ![Comparison table of Snakemake and Nextflow across pipeline language, Batch support, storage, environment control and pipeline collections.](/article-images/illustrations/snakemake-google-cloud-life-sciences-nextflow-or-snakemake.webp) Both are mature bioinformatics workflow managers, and both run well on the same GCP primitives, Batch VMs and Cloud Storage. The differences are in language, packaging, and ecosystem. | Aspect | Snakemake | Nextflow | |, - |, - |, - | | Pipeline language | Python-based DSL of rules and wildcards | Groovy-based DSL2 of processes and channels | | Google Batch support | Official executor plugin, installed with pip | Built-in executor in the core distribution | | Cloud storage | Storage plugin system handling `gs://` paths | Native cloud storage path handling in the core | | Environment control | Conda env or container per rule | Container per process, conda also supported | | Shared pipeline collections | Snakemake workflow catalog | nf-core collection | | Natural fit | Python teams, file-pattern-driven pipelines | Teams already invested in DSL2 or nf-core | Choose Snakemake if your team writes Python, thinks in file patterns, or wants fine-grained control over what reruns. Choose Nextflow if your team already maintains DSL2 code or draws from nf-core pipelines. Converting between them is a rewrite, not a port, so the switching cost almost always exceeds the marginal difference on Google Cloud. Both give you the same fundamental outcome, per-step VM execution with bucket-backed data. ## Compliance, validation, and reproducibility Life sciences pipelines often feed regulated work, and moving them to managed cloud does not move the validation burden off your shoulders. Conventional computer system validation was built for static on-premises systems and does not handle the dynamic, scalable, distributed character of cloud platforms [6]. One proposed answer is a Cloud Compliance Intelligence Framework that pairs intelligent automation with risk-based compliance so validation remains scalable and auditable as the platform changes [6]. Translated into Snakemake practice, that means concrete habits. Pin tool versions in env files or container tags. Record every executed command with `-printshellcmds` and keep the logs. Scope IAM to a dedicated service account with bucket-level access rather than broad project roles. Treat the `.snakemake/` metadata as your change record and keep it under version control. Revalidate when you bump a version or change a resource profile. Provider security measures are part of what draws life sciences work to GCP in the first place [3], but the auditable process around your pipeline is yours to build and keep. ## Frequently Asked Questions ### Can I still use the old Google Life Sciences executor in Snakemake? No. Google retired the Cloud Life Sciences API in 2024 and Snakemake 8 removed the executor during its plugin reorganization. Use the Google Batch executor plugin, which keeps the same architecture of one VM job per rule with Cloud Storage as the shared file system. ### What replaced Google Cloud Life Sciences for Snakemake users? Google Batch plus two packages, the `snakemake-executor-plugin-googlebatch` executor and the `snakemake-storage-plugin-gcs` storage provider. You enable `batch.googleapis.com` instead of the retired Life Sciences API, and everything else in your Snakefile stays the same. ### Do I need containers to run Snakemake on Google Batch? Containers are not mandatory but strongly recommended. Conda environments are rebuilt on every job VM, which adds minutes per job, while an Apptainer image pulls once and starts immediately. Use `-use-conda` for quick tests and `-use-apptainer` for production runs. ### Where does the snakemake command itself run? On any machine with the gcloud credentials and the plugins installed, your laptop, a small VM, or a CI runner. The head node only builds the DAG, submits jobs, and polls status, so it needs almost no compute and no local copy of the data. ### Nextflow or Snakemake for Google Cloud, which should I choose? Both run well on Google Batch and Cloud Storage. Pick Snakemake for Python-style rules, wildcard-driven file handling, and precise reruns. Pick Nextflow if your team prefers Groovy DSL2 or already uses nf-core pipelines. Switching later is a rewrite, so decide based on team skills and existing code. ### How do I resume a cloud run that failed halfway? Run the exact same command again. Snakemake skips jobs whose outputs exist and are recorded as complete, resubmits the rest, and `-rerun-incomplete` cleans up partial files. Check the rule logs in the bucket and the Batch task output in Cloud Logging to fix the underlying cause first. ### Can I keep the run private and compliant in this setup? Yes. Use a dedicated service account with least-privilege roles, uniform bucket-level access, and a single region. For regulated GxP work, the validation duty stays with you, so pin versions, log every command, and keep the workflow metadata as an auditable record. ### What limits will I hit first when scaling up? Regional vCPU quotas and API rate limits. A run requesting more CPUs than the regional quota allows fails with a quota error, so either lower `-jobs`, request an increase, or choose a less busy region. Throttle status polling with `-max-status-checks-per-second` on very large DAGs. ## Sources 1. [Implementing Cloud Computing In Small & Mid-Market Life-Sciences](https://doi.org/10.58729/1941-6679.1015) 2. [Implementing Cloud Computing in Small and Mid-Market Life-Sciences: A Mixed-Method Study](https://www.semanticscholar.org/paper/9b838dcfdfd9b3ecc9d977fc7ef81ac226d4c38b) 3. [Harnessing Cloud Synergy for Streamlined Healthcare and Life Sciences Ventures](https://doi.org/10.1109/ICACRS58579.2023.10404275) 4. [Emerging Computational Methods for the Life Sciences Workshop Special Issue](https://doi.org/10.1002/cpe.1862) 5. [Special Issue for Emerging Computational Methods for the Life Sciences Workshop](https://doi.org/10.1002/cpe.2998) 6. [CLOUD COMPLIANCE INTELLIGENCE FRAMEWORK FOR GXP VALIDATION IN LIFE SCIENCES](https://doi.org/10.12732/ijam.v38i11s.1214)