# Nextflow Resource Optimization: How to Diagnose Most Nextflow runs that need resource optimization share one root cause: the resources a task requests do not match the resources the task actually uses. A process asks for 2 GB of memory, the aligner peaks at 12 GB, and the Linux out-of-memory killer terminates the task with exit code 137. Or the reverse happens. Every task asks for 16 CPUs, the scheduler can only place two tasks at a time, and a pipeline that could finish in an hour runs all weekend. Nextflow records the submission time, start time, completion time, CPU usage, memory usage, and disk usage for every task it runs, which makes this class of problem measurable rather than guessable [1]. This guide is a diagnostic manual. It walks through the first checks to run, the mechanism behind each common failure, a symptom table you can scan against your own run, and the concrete fixes that follow from each diagnosis. The examples lean on RNA-seq and bacterial genomics because those are the workflows where people hit these walls most often, but the diagnostic sequence applies to any Nextflow pipeline, from bacterial assemblies [2] to metagenome-assembled genome recovery [3] to multi-omics neoantigen prediction [4]. ## Start Here: The First Ten Minutes of Diagnosis ![Decision path asking which tasks failed, how much resource they used, and whether the run failed, stalled or ran slowly, then the first check to run.](/article-images/illustrations/nextflow-resource-optimization-triage-first-checks.webp) Before touching any config file, answer three questions from your most recent failed or slow run. **Question one: which tasks failed, and with what exit code?** The exit code is the single most informative piece of data in bioinformatics troubleshooting for workflow failures. An exit code of 137 means the task was killed by signal 9, which in practice almost always means the host or scheduler ran out of memory and killed the process. An exit code of 139 means a segmentation fault inside the tool. An exit code of 143 means the task was terminated, often by a scheduler time limit. Nextflow also prints a status line for every task in its main log, so one command usually identifies the culprit: ```bash grep "exit status" .nextflow.log | tail -n 20 ``` **Question two: how much resource did the failed task actually use before it died?** Nextflow can write a trace file with one row per task. If you did not enable it at launch time, you can still inspect the `.command.err`, `.command.out`, and `.command.run` files inside the failed task's work directory, which Nextflow prints the path to in the error message. **Question three: did the run fail, stall, or merely run slowly?** These are different diseases. A failure has an exit code and a stack trace. A stall shows many submitted tasks and few running ones. Slow execution shows tasks completing successfully but taking far longer than the same tool would standalone. Diagnose the category first, then the specific cause. If your pipeline produces a resource report automatically, read it before you change anything. Web-based tools such as Nextpie collect these records into a database and visualize the resource profile of a whole pipeline, which turns optimization from anecdote into measurement [1]. ## How Nextflow Requests Resources To diagnose resource problems you need a mental model of what Nextflow actually does. Nextflow is a workflow manager, not a scheduler. When a process is ready to run, Nextflow generates a job script from the process definition and hands it to an executor. On a laptop that is the local executor, which runs the script as a child process. On a cluster it is an executor for the site's batch system, such as SLURM, which submits the script to a queue. In the cloud it submits to a container service. The executor decides where the task physically runs. Nextflow only decides what to ask for. The ask comes from process directives. The three that matter most are `cpus`, `memory`, and `time`: ```groovy process ALIGN { cpus = 8 memory = 32.GB time = 12.h script: """ hisat2 -p ${task.cpus} -x index -1 reads_1.fq.gz -2 reads_2.fq.gz | samtools sort -@ ${task.cpus} -o sorted.bam """ } ``` Two failure modes live inside this simple picture. First, if the requested values are lower than what the tool consumes, the task gets killed. Second, if the requested values are higher than what the tool consumes, the scheduler holds the task until a machine with that much free capacity is available, and the pipeline starves in the queue. Both look like "the pipeline is broken" from the outside, and both are fixed by closing the gap between requested and actual usage. A third failure mode is a disconnect between the directive and the tool itself. If `cpus = 8` but the script never passes `${task.cpus}` to the tool, the tool runs single-threaded while occupying eight cores. The pipeline is correct but wasteful, and every downstream task waits longer than it should. Closing that gap is a central theme in published pipeline engineering. When the authors of MTBseq-nf wrapped the original MTBseq pipeline for tuberculosis genomics, their central improvement was allowing several instances of the same step to run in parallel so the available resources were fully utilized, which made their wrapper at least twice as fast as the standard pipeline for cohorts exceeding 20 samples [5]. The hardware had not changed. The match between workflow structure and available resources had. ## The Core Troubleshooting Table Scan this table against your symptoms. The "how to confirm" column tells you exactly what evidence to collect before changing anything. | Symptom | Likely cause | How to confirm | Fix | |--|--|--|--| | Task dies with exit code 137, "Killed" in the log | Host or scheduler out of memory, task killed by the OOM mechanism | Check `.nextflow.log` for the exit status, read `.command.err`, compare `peak_rss` in the trace file with the `memory` directive | Raise the `memory` directive for that process, ideally through a label, and add a retry rule | | Task dies with exit code 139 or 134 | Genuine crash inside the tool, segfault or failed assertion | Read the stack trace in `.command.err` | Reproduce the command manually, report upstream, pin the container version, test with reduced thread counts | | Pipeline submits hundreds of tasks but few run | Per-task requests exceed what any free node can offer | Compare the `cpus` and `memory` columns of the trace file against your queue's node sizes | Reduce per-task requests, split large inputs, or route large tasks through a high-memory label | | Tasks complete but each one is far slower than the tool run manually | Tool threads do not match allocated CPUs, or tasks steal each other's cores | Compare the `%cpu` column in the trace with the requested `cpus` | Pass `${task.cpus}` into every tool, cap concurrent tasks with `maxForks` | | Whole run is slow, and the head node feels sluggish | Work directory on a slow or contended filesystem, or Nextflow running on a shared login node | Check where `workDir` points, measure read and write speed on that mount | Move `workDir` to fast scratch storage, run Nextflow from a dedicated submission node | | Your `memory` and `cpus` settings appear to be ignored | Config file not loaded, wrong profile selected, or directives placed in the wrong scope | Run `nextflow config` and inspect the resolved values, then read `.command.run` for one task | Correct the config path, use the intended profile, move directives inside the `process` scope | | Pipeline stalls midway with many cached tasks and few active ones | Concurrency limits, scheduler account limits, or tools locking shared databases | Count submitted versus running tasks in `.nextflow.log`, look for scheduler hold messages | Raise executor queue limits where permitted, stagger tasks that share a locked resource | | Run fails at launch with a configuration error | Typo or misplaced directive in `nextflow.config` | `nextflow config` reproduces the error without running the pipeline | Fix the directive name or scope | | Long stretches where nothing runs except downloads | Container images being pulled per task with no shared cache | Look for repeated pull lines in `.nextflow.log` | Pre-pull images to a shared cache, or use a local registry | | Reruns give different runtimes and occasionally different failures | Non-deterministic thread behavior or temp directory collisions | Diff `.command.sh` between two task directories for the same process | Pin thread counts explicitly, give each task a unique temporary directory | One caution before you begin fixing things: never clean up with a blanket recursive delete against the work directory while a run is active or until you have confirmed your outputs are safe elsewhere. The work directory holds the only copy of intermediate files, the task scripts, and the logs you need for diagnosis. Deleting it destroys both your resume capability and your evidence. ## Causes, From Most Common to Rarest ### Memory under-allocation, the default diagnosis When a Nextflow task fails on a cluster, memory is the first suspect. The mechanism is specific. The task runs inside the memory limit the scheduler granted. The tool exceeds that limit. The kernel's out-of-memory mechanism selects the largest memory consumer, which is your task, and sends it signal 9. The exit code becomes 137, the error message is terse, and the only durable evidence is in the log files. The diagnostic habit that saves the most time is recording actual peak memory for every task from day one. Launch with the built-in reporting flags: ```bash nextflow run mypipeline -with-trace -with-report -with-timeline -resume ``` The trace file contains a `peak_rss` column, the peak resident set size the task reached. Compare that number with the `memory` directive for the process. If the task succeeded and `peak_rss` sits within a few percent of the request, that process is under-provisioned for slightly larger inputs and will fail on your next cohort. If the task died, look at how far through its runtime it got. A task killed early was probably killed at startup allocation, while a task killed late hit a data-dependent peak, for example a sorter buffering a large BAM file. The usual fixes, in order of preference, are a labeled tier for heavy processes, a dynamic retry rule that grows resources with each attempt, and input splitting so no single task has to hold an unusually large object. All three are covered in the fixes section below. ### Over-requesting and queue starvation The mirror-image failure gets less attention because it produces no error at all. Every task asks for 16 CPUs and 64 GB. Your partition has 40 nodes with 16 cores each, most already busy. The scheduler can place two of your tasks at a time. The pipeline appears healthy, tasks succeed one by one, and a run that should take three hours takes two days. There is no error to grep for, which is why this cause is diagnosed from the trace file rather than the error log. Sort the trace by runtime and look at the gap between the `submit` and `start` timestamps. Large, systematic gaps mean tasks are waiting in the queue, not computing. Then compare per-task requests against the smallest node class you are allowed to use. The remedy is almost always finer granularity. Ask many tasks for modest resources rather than few tasks for everything. This is the same insight that drove MTBseq-nf, where replacing a linear, batched analysis with per-step parallelization let the available resources be fully utilized and roughly doubled throughput on real cohorts [5]. Input granularity matters here too. [Metagenome assembly](/knowledge/bioinformatics/de-novo-metagenome-assembly-a-comprehensive-guide-to-algorithms-and-workflows) is the canonical example, because de novo assembly is the step that consumes the most computational time and resources in MAG recovery workflows. When the authors of MetaBolt tackled this, their approach was not to request bigger nodes but to reduce the work per node, using an empirically optimized set of k-mers for MEGAHIT-based assembly. The result was recovery of metagenome-assembled genomes at 2.3 times the rate of nf-core/mag and 3.8 times the rate of MetaWRAP, with roughly 5 percent more high-quality MAGs, and computational times reduced to 2 to 4 percent of baseline for low-biomass real metagenome samples [3]. Faster came from smarter task sizing, not bigger hardware. ### CPU oversubscription and silent throttling A task can be memory-healthy and still waste hours. The common pattern is a directive that reserves 8 CPUs for a task whose script never tells the tool to use more than one thread, or the opposite, a tool that spawns threads without limit inside a 4-CPU allocation. In the first case you pay for idle cores. In the second case the scheduler's core accounting no longer reflects reality, threads contend, and every task on the node slows down. Confirm this from the trace file's CPU utilization column. A task allocated 8 CPUs that averages 12 percent utilization is single-threaded despite its allocation. A task averaging well over 100 percent of its allocation is oversubscribed. The fix for the first is mechanical. Thread-aware tools should receive the allocation explicitly, for example `salmon quant -p ${task.cpus}` or `samtools sort -@ ${task.cpus}`. The fix for the second is `maxForks`, which caps how many instances of a process run concurrently, useful for tools that allocate per-thread buffers proportional to machine size rather than to their allocation. ### Disk I/O and the work directory Every Nextflow task stages its input files into a work directory and writes outputs there. On a busy shared filesystem, hundreds of concurrent tasks staging and writing simultaneously turn the filesystem into the bottleneck, and every task slows down together. The symptom is uniform slowness across unrelated processes, high wait time rather than high user time, and a work directory mount that feels laggy during interactive use. Confirm by checking where `workDir` points and testing throughput on that mount while the pipeline runs. The fix is to place the work directory on fast scratch storage local to the compute nodes, or on a parallel filesystem designed for concurrent access, and to keep long-term results in a separate, slower, durable location. On cloud and HPC systems, workflow structure should follow storage topology. That consideration is one reason portability across execution environments is a recurring design goal in published Nextflow pipelines, whether for population genomics run locally or on high-performance computing systems [6] or for automated neoantigen discovery built as a scalable, portable Nextflow workflow [4]. ### Configuration that never took effect A frequent and humiliating cause: you spent an hour tuning `nextflow.config`, and the run never read it. Perhaps the file was edited after launch, perhaps the profile name was misspelled, perhaps an `includeConfig` path pointed to a stale copy, or perhaps the directives were placed outside the `process` scope so Nextflow treated them as unknown tokens. The confirmation is cheap and definitive. Run: ```bash nextflow config ``` This prints the fully resolved configuration without running anything. Then open one task's `.command.run` file in the work directory and find the resource values the scheduler actually received. If the resolved config says 32 GB and the job script says 2 GB, something between the config and submission overrode it, most commonly a command-line parameter or a second config file loaded later in the include chain. ### Parallelism limits and structural bottlenecks Sometimes resources are fine and the workflow structure is the constraint. Three variants appear repeatedly. First, `maxForks` or executor queue limits hold concurrency down deliberately. This is correct behavior for tools that cannot run concurrently, such as some database-backed annotation steps, but it is a silent cap if you inherited it in a config and forgot. Second, a funnel shape in the DAG. The pipeline runs 90 samples through QC in parallel, then reaches a step that merges everything into one task. That single task becomes the wall clock bottleneck no matter how many cores the cluster offers. The published remedy is decomposition. Bactopia's design separates per-sample analysis from downstream comparative steps, with modular tools for specific postprocessing such as pan-[genome analysis](/blog/guides/genome-analysis) or average nucleotide identity, so the per-sample layer stays embarrassingly parallel across collections that can scale from a single genome to thousands, as demonstrated on 1,664 public Lactobacillus genomes [2]. Third, parameters that multiply work. Running a pipeline under many parameter settings requires the workflow system to track and execute each combination without redundant recomputation. Frameworks such as JUDI address this by attaching a parameter database to each task and file, reducing the scripting burden for n parameters by a factor on the order of n and making parameter exploration systematic rather than ad hoc [7]. If your optimization problem is "I need to sweep thresholds and each sweep costs a full run," the solution is parameter-aware design, not bigger nodes. ### Software environment overhead The rarest tier of causes lives in the software layer. Container images pulled fresh for every task add minutes per task and saturate the network. Conda environment creation inside a task can take longer than the analysis. On filesystems without a shared image cache, this overhead dominates short tasks entirely. The remedy is pre-pulling images into a shared cache or using a local registry, a practice embedded in how containerized pipelines are deployed across Conda, Singularity, and Docker environments [6]. Nanopore-focused workflows illustrate a related environment burden. SquiDPipe, the Nextflow pipeline behind the SquiDBase microbial nanopore repository, performs automated removal of human reads from raw nanopore data, a step whose runtime scales with raw signal volume rather than with final sequence output, so environment and I/O efficiency weigh heavily in its design [8]. ## A Diagnostic Command Reference Keep this sequence on hand. It moves from coarse to fine. **1. Enable observability at launch.** Every serious run should start with the trace, report, and timeline enabled: ```bash nextflow run mypipeline -with-trace -with-report -with-timeline -resume ``` The trace is a CSV with one row per task. The report is an HTML summary of resource usage. The timeline shows when each task ran, which makes queue gaps and funnel bottlenecks visible directly. **2. Find the failing tasks.** Filter the run history by status: ```bash nextflow log -f status=FAILED ``` And list the fields that matter for a specific historical run by its session ID: ```bash nextflow log -f process,task_id,status,exit,peak_rss,%cpu ``` **3. Autopsy one failed task.** Navigate into the work directory Nextflow prints in its error message. Five files tell the whole story. `.command.sh` is the exact script that ran. `.command.err` holds the tool's stderr. `.command.out` holds stdout. `.command.run` is the full job script including the resource requests the scheduler received. `.exitcode` records the exit code. Reading `.command.err` first is the habit that separates ten-minute diagnoses from ten-hour ones. **4. Check the scheduler side.** On HPC systems, the scheduler's own accounting for the job shows whether the task was killed by the scheduler for exceeding its request or died inside the allocation. A task killed for exceeding its request is a config problem. A task killed while inside its request, with the tool reporting a memory error, is a tool-level problem such as a sorter whose maximum memory setting exceeded the allocation. ## RNA-seq Error Patterns RNA-seq workflows are where most people meet resource failures for the first time, and the failures cluster into recognizable patterns. The heaviest single step is usually building the aligner index. Splice-aware aligners for eukaryotic genomes need index-generation memory in the tens of gigabytes, which is an established property of the tools, so an index-build task under a small default allocation is a classic exit code 137 generator. The task also runs once for the whole project, which means it is easy to miss in per-sample testing and then fails on the first full run. Give index-building processes their own high-memory label. The second pattern is large sorted output. Alignment produces SAM-scale text or BAM files, and sorting a deep RNA-seq sample's BAM requires the sorter's memory ceiling to fit inside the task allocation. If the sorter's configured limit exceeds the task's requested memory, the task dies precisely when the sorter's buffer fills, late in the run, which makes it look intermittent. The third pattern is the aggregation tail. Per-sample steps finish quickly in parallel, then the run appears to hang on a final task that reads every sample's output, such as report generation. This is the funnel shape described above, and the fix is to accept a single longer allocation for that task rather than trying to parallelize an inherently single-task step. The fourth pattern is retry masking. A dynamic retry rule that doubles memory on the second attempt turns a hard failure into a slow success. That is useful, but if you never look at the trace you will never learn that your first-attempt allocations are wrong, and every full run pays the wasted attempt. Treat retries as a safety net, not a tuning strategy. ## Profiling at Scale Single-run debugging fixes one run. Profiling fixes the pipeline. Nextflow writes resource records for each task, but they land in log files scattered across a filesystem, and aggregating them manually across many runs quickly becomes cumbersome. Nextpie addresses exactly this by loading the records into a relational database and providing web-based reporting and interactive visualization of a pipeline's resource usage, with source code, documentation, and a test database available on GitHub [1]. The practical profiling loop looks like this. Run the pipeline on a representative input cohort with the trace enabled. Import or aggregate the trace data. Identify the processes with the widest gap between requested and peak resources. Adjust the directives for those processes only. Re-run on the same cohort and compare. Two or three cycles of this usually bring peak resource usage within a comfortable margin of requests across the whole pipeline. When you profile, profile on your real data and your real cohort size. Benchmarks published for pipeline comparisons consistently use defined cohorts, for example the 90 Mycobacterium tuberculosis genomes used to evaluate MTBseq-nf scalability and reproducibility [5]. A pipeline tuned on two tiny test files will mispredict behavior on 90 real samples, because per-task memory often scales with input size in ways small tests never trigger. ## How to Fix What Diagnosis Finds ![Worked calculation showing a task starting at 4 GB and 4 CPUs, doubling to 8 GB and 8 CPUs on the second attempt, with maxRetries set to 2.](/article-images/illustrations/nextflow-resource-optimization-retry-growth.webp) Once a cause is confirmed, the fix is usually one of a small set of patterns. **Use labels to tier your processes.** Define resource tiers once, in a shared config, and assign each process to a tier: ```groovy process { withLabel: 'low' { cpus = 2 memory = 4.GB time = 4.h } withLabel: 'highmem' { cpus = 8 memory = 64.GB time = 24.h } } ``` Labels keep resource policy in configuration rather than scattered through process definitions, and they let cluster-specific profiles override the numbers without touching the pipeline itself. **Add a growth-on-retry rule.** A standard pattern lets failing tasks return with more resources rather than failing the run: ```groovy process { errorStrategy = 'retry' memory = { 4.GB * task.attempt } cpus = { 4 * task.attempt } maxRetries = 2 } ``` On the first attempt the task gets 4 GB and 4 CPUs. On a retry it gets 8 GB and 8 CPUs. The `task.attempt` variable is what makes the growth automatic. Combine this with truthful first-attempt values so retries stay rare. **Feed the allocation to the tools.** Every multithreaded tool in every script should reference `${task.cpus}`. Grep your processes for hardcoded thread numbers. Each one is either a wasted allocation or a latent oversubscription. **Split what cannot be shrunk.** If one task's memory requirement is inherently large, ask whether the input can be partitioned, whether the tool supports chunked operation, or whether the step can be restructured into per-partition tasks followed by a merge. MetaBolt's gains came from exactly this kind of step-level optimization of the most expensive stage rather than from adding hardware [3]. **Match the executor to the workload.** Short, many-task pipelines behave differently from long, few-task pipelines across executors. Population genomic workflows such as scalepopgen are built to run either locally or on HPC systems, with container or environment choices made per deployment [6]. Revisit that choice when the diagnosis points at submission overhead or queue behavior rather than at any individual task. ## What the Literature Adds It helps to see these principles working in published pipelines. MTBseq-nf demonstrates the parallelization principle, replacing a batched, linear execution model with per-step task parallelism and roughly doubling speed on cohorts beyond 20 samples [5]. MetaBolt demonstrates step-level optimization, targeting assembly, the dominant cost in MAG recovery, and achieving multiple-fold speedups over comparable workflows while improving output quality [3]. Bactopia demonstrates modular scaling, separating per-sample processing from comparative tools so a bacterial pipeline scales smoothly from one genome to cohorts in the thousands [2]. OmniNeo demonstrates integration at scale, combining WGS, WES, transcriptomic, and proteomic data within a single Nextflow workflow for neoantigen prediction [4]. SquiDBase's SquiDPipe demonstrates resource-aware design under data-volume pressure, processing raw nanopore signal for human read removal across a community data resource [8]. JUDI demonstrates that some resource optimization is really parameter-management optimization, reducing the scripting needed to explore parameter settings by an order proportional to the parameter count [7]. And Nextpie demonstrates the measurement layer, turning scattered per-task records into queryable, visualizable resource profiles [6] for routine use [1]. The common thread is that none of these groups optimized by guessing. They measured, identified the dominant cost, and changed the structure of the work. ## Common Mistakes and Limitations **Tuning before measuring.** Changing memory values based on a hunch, without a trace file, usually trades one failure mode for another. Get the `peak_rss` numbers first. **Tuning to the test dataset.** A resource profile that fits a two-sample smoke test will not fit a 90-sample cohort. Memory in sorters and assemblers scales with data depth, so validate allocations on data at production scale or expect to retune. **Treating a retry as a fix.** If tasks consistently need a second attempt, your first-attempt allocations are wrong. Retries cost time and queue slots on every run. **Blaming the scheduler for everything.** Before filing a support ticket, confirm your job actually requested what you think it requested by reading `.command.run`. A large fraction of "the cluster is misconfigured" reports end with a config file that was never loaded. **Optimizing the wrong process.** The loudest failure is not always the expensive one. A process that fails dramatically might run once, while a modest process running 500 times dominates total wall time. Sort your trace by cumulative resource use, not by failure count. **Assuming resource errors are deterministic.** A task that fails with exit code 137 on one node may pass on a node with more free memory, because the host-level OOM behavior depends on co-located load. Flaky memory failures are still memory failures, and the fix is the same. **Ignoring the limitation of local evidence.** Trace metrics reflect what the task consumed where it ran. Moving a pipeline to a different filesystem, container runtime, or node type can shift the profile, so re-profile after any major environment change. ## Prevention Practices Prevention in Nextflow resource management is mostly a matter of instrumentation and review habits that cost minutes and save days. Instrument every run by default. Put the trace, report, and timeline flags into a standard launch wrapper or profile so no run goes unmeasured. Nextflow already records the underlying data for each task [1], and a reporting layer over it makes the difference between noticing drift next run and noticing it next quarter. Keep resource policy in versioned config files, organized by label and profile, and never hardcode thread counts inside process scripts. Review the resource profile after any change to input scale. A pipeline that handled 20 samples gracefully may need its high-memory tier adjusted at 200. Structure pipelines to stay parallel where possible. Per-sample steps should be per-sample tasks, aggregation steps should be explicit and few, and heavy single tasks should be isolated behind labels so they do not force the whole pipeline onto large node classes. Finally, cultivate the autopsy habit. When a task fails, read `.command.err` before anything else, then the exit code, then the trace row. Most resource failures explain themselves in under five minutes to someone who reads the evidence in the right order. ## Frequently Asked Questions ### What does exit code 137 mean in a Nextflow task? It means the task process received signal 9, which in practice almost always means the operating system or scheduler killed the task because it exceeded its memory. Confirm by reading `.command.err` in the task's work directory and by comparing the task's actual peak memory with the `memory` directive, then raise the allocation. ### How do I see how much memory each Nextflow task actually used? Launch with `-with-trace` and open the resulting trace file, which contains a `peak_rss` column with the peak resident set size per task. For runs that already finished, aggregated views of the same records can be built with reporting tools such as Nextpie. ### Why is my Nextflow pipeline submitting tasks but nothing is running? This usually means the scheduler cannot place your tasks because their per-task CPU or memory requests exceed what free nodes offer, or your account's concurrent job limit is reached. Compare the requested resources in the trace file against your queue's capacity and lower the requests or split large tasks. ### Should I request more CPUs to make my pipeline faster? Only if the tools actually use the threads. Passing the allocation into each tool through a variable tied to the task's CPU count is what converts a request into speed, and tasks that hold idle cores block other tasks from starting. ### How do I rerun only the failed tasks from a previous run? Use the `-resume` flag with the failed run's session ID. Nextflow marks successful tasks as cached and re-executes only the failed ones, provided their inputs and code have not changed. ### What is the fastest way to find the failing step in an RNA-seq run? Check the exit codes with `nextflow log` filtered by failed status, open the work directory of one failed task, and read `.command.err` first. In RNA-seq workflows, look hardest at index-building and BAM-sorting steps, which are the most memory demanding. ### Do I need to delete the work directory to fix resource errors? No. The work directory holds your task scripts, logs, and intermediates needed for resume, and deleting it destroys your evidence. Adjust the resource directives and resume the run instead. ### How much memory should I request for a task by default? Start from measurement rather than a universal number. Run a representative cohort with the trace enabled, take the observed peak memory for each process, add a comfortable margin, and encode those values under labels in your config. ## Sources 1. [Nextpie: a web-based reporting tool and database for reproducible nextflow pipelines.](https://pubmed.ncbi.nlm.nih.gov/41180966/) 2. [Bactopia: a Flexible Pipeline for Complete Analysis of Bacterial Genomes.](https://pubmed.ncbi.nlm.nih.gov/32753501/) 3. [MetaBolt: A computationally efficient pipeline for the rapid recovery of metagenome-assembled genomes.](https://pubmed.ncbi.nlm.nih.gov/41830759/) 4. [OmniNeo: a multi-omics pipeline incorporating proteomics and AI selection for neoantigen optimization in tumor immunotherapy.](https://pubmed.ncbi.nlm.nih.gov/41479904/) 5. [MTBseq-nf: Enabling Scalable Tuberculosis Genomics "Big Data" Analysis Through a User-Friendly Nextflow Wrapper for MTBseq Pipeline.](https://pubmed.ncbi.nlm.nih.gov/41471889/) 6. [scalepopgen: Bioinformatic Workflow Resources Implemented in Nextflow for Comprehensive Population Genomic Analyses.](https://pubmed.ncbi.nlm.nih.gov/38507648/) 7. [Bioinformatics pipeline using JUDI: Just Do It!](https://pubmed.ncbi.nlm.nih.gov/31882996/) 8. [SquiDBase: a community resource of raw nanopore data from microbes.](https://pubmed.ncbi.nlm.nih.gov/41524028/)