# Deconvoluting Spatial Transcriptomics Spots: A Comparison of Cell2location, RCTD, and SPOTlight ## Key Takeaways - Cell2location, RCTD, and SPOTlight are reference-based deconvolution methods for spatial transcriptomics, each employing distinct modeling approaches: Cell2location utilizes a Bayesian hierarchical model, RCTD uses penalized least squares, and SPOTlight relies on non-negative matrix factorization with marker gene selection. - Performance varies significantly by tissue type; Cell2location demonstrated the highest average performance across cardiovascular and kidney disease experiments, while RCTD achieved superior accuracy in cardiovascular samples, underscoring the need for tissue-specific validation. - Computational intensity differs: RCTD is the fastest with low to moderate requirements, Cell2location is computationally intensive and often requires GPU acceleration, and SPOTlight falls in the moderate range. - Reference data requirements vary: Cell2location converges with less reference data, whereas RCTD necessitates more comprehensive reference datasets for stable proportion estimates. - Quality assessment of deconvolution results is critical and should include proportion sum checks, examination of cell-type proportion distributions, spatial pattern assessment, and comparison with histological annotations to identify potential biases or errors. - Reproducibility necessitates meticulous documentation of the reference dataset, method versions and parameters, computational environment, and output storage, alongside adherence to community standards for workflow configuration. --- Spatial transcriptomics platforms such as 10x Visium capture whole-transcriptome expression at spot-level resolution, where each measurement aggregates transcripts from multiple cells within a capture area. Researchers must estimate cell-type proportions within each spot to interpret tissue architecture, but the choice of deconvolution tool materially affects downstream biological conclusions. This article compares three widely used reference-based deconvolution methods, Cell2location, RCTD, and SPOTlight, using evidence from published benchmarks and methodological reviews. The practical outcome is a decision framework for selecting a deconvolution tool based on tissue type, reference data availability, computational resources, and the biological question at hand. The comparison draws on a large benchmarking study that evaluated 21 deconvolution methods across 37 datasets spanning brain, cancer, and organ tissues and four distinct spatial technologies, identifying Cell2location, RCTD, and SONAR as top-performing methods while noting substantial performance variation across tissue types. Additional evidence from cardiovascular and kidney disease samples shows that all three methods perform comparably well for verifiable cell types, with RCTD achieving the best accuracy in cardiovascular samples and Cell2location achieving the highest average performance across all test experiments. The practical implications for laboratory professionals and researchers are presented below, including workflow decisions, quality controls, and common failure patterns. ## At a Glance: Method Comparison for Spot Deconvolution | Feature | Cell2location | RCTD | SPOTlight | | --- | --- | --- | --- | | Core modeling approach | Bayesian hierarchical model with negative binomial distribution | Robust cell-type decomposition using penalized least squares | Non-negative matrix factorization with marker gene selection | | Reference data requirement | Single-cell or single-nucleus RNA-seq reference | Single-cell or single-nucleus RNA-seq reference | Single-cell or single-nucleus RNA-seq reference | | Computational intensity | High, requires GPU for large datasets | Low to moderate, fastest among the three | Moderate | | Reference data needed for convergence | Less reference data needed to converge | More reference data needed for stable results | Marker genes must be identified from reference | | Reported performance | Highest average performance across cardiovascular and kidney disease test experiments | Best accuracy in cardiovascular disease samples | Not included in the cardiovascular and kidney benchmark | | Strengths | Handles technical variation between platforms, estimates cell-type proportions with uncertainty | Simple workflow, fewer computational dependencies, fast runtime | Useful for estimating proportions when marker genes are well characterized | | Limitations | Higher computational intensity, longer runtime | Requires sufficient reference data for convergence | Performance depends on marker gene selection quality | ## Understanding Spot-Level Resolution and the Deconvolution Problem Sequencing-based spatial transcriptomics divides tissue sections into capture spots, each containing a mixture of cells. The measured expression at each spot represents an aggregate of transcripts from all cells within that capture area. This aggregation obscures the underlying cellular composition, creating the need for computational deconvolution to estimate the proportion of each cell type present at each spatial location. The resolution limitation is inherent to platforms such as Visium and Stereo-seq, which offer transcriptome-wide discovery but capture multiple cells per spot. Imaging-based methods such as MERFISH, Xenium, and CosMx provide subcellular resolution with higher sensitivity, but they require predefined gene panels and do not offer whole-transcriptome coverage. For researchers using sequencing-based platforms, deconvolution is a required analytical step before cell-type-specific spatial patterns can be interpreted. The deconvolution problem has two main components. First, the method must estimate the cell-type composition at each spot. Second, the method must account for technical differences between the single-cell RNA sequencing reference and the spatial transcriptomics data, including differences in sequencing depth, capture efficiency, and platform-specific biases. Methods that fail to account for these technical differences produce biased estimates that do not reflect true tissue composition. Reference-based deconvolution methods require an external single-cell or single-nucleus RNA-seq dataset that defines the expected expression profile of each cell type. The quality of this reference directly determines the quality of the deconvolution output. A reference that lacks a relevant cell type, contains batch effects, or uses a different tissue state will produce misleading proportion estimates. Reference-independent methods exist, such as STEA, which uses enrichment-based annotation without requiring single-cell RNA-seq data, but these methods are not the focus of this comparison. ## Core Principles of Reference-Based Deconvolution Reference-based deconvolution methods share a common conceptual framework. They use a single-cell RNA-seq dataset to define cell-type-specific expression profiles, then estimate the mixture proportions at each spatial spot that best explain the observed spot-level expression. The methods differ in their statistical models, optimization strategies, and how they handle technical variation. ### Cell2location: Bayesian Hierarchical Modeling Cell2location uses a Bayesian hierarchical model with a negative binomial distribution to estimate cell-type proportions. The model accounts for technical variation between the single-cell reference and the spatial data by learning platform-specific effects. This approach allows Cell2location to estimate the proportion of each cell type and the uncertainty associated with each estimate. The Bayesian framework provides several practical advantages. Cell2location can incorporate prior knowledge about tissue composition, and it produces posterior distributions that quantify estimation uncertainty. This uncertainty information is valuable for downstream analyses because it allows researchers to identify spots where the deconvolution result is reliable and spots where it is not. The main practical limitation of Cell2location is computational intensity. The model requires iterative optimization, and for large spatial datasets, a GPU is often necessary to achieve reasonable runtime. The benchmark study on cardiovascular and kidney disease samples found that Cell2location needed less reference data to converge but required higher computational intensity compared with RCTD. ### RCTD: Robust Cell-Type Decomposition RCTD uses a robust cell-type decomposition approach based on penalized least squares. The method estimates cell-type proportions by fitting the observed spot-level expression to a linear combination of cell-type-specific expression profiles. The penalization helps to stabilize estimates when cell types have similar expression profiles. RCTD is designed to be computationally efficient and has the simplest workflow among the three methods. The benchmark study found that RCTD had the fastest computational time and required fewer computational dependencies. This makes RCTD an attractive option for researchers who need quick results or who lack access to high-performance computing resources. The main limitation of RCTD is its requirement for sufficient reference data. The method needs enough cells per cell type in the reference to produce stable expression profiles. When the reference dataset is small or when rare cell types are underrepresented, RCTD may produce unstable or biased estimates. ### SPOTlight: Non-Negative Matrix Factorization with Marker Genes SPOTlight uses non-negative matrix factorization combined with marker gene selection to estimate cell-type proportions. The method first identifies marker genes for each cell type from the reference, then uses these markers to guide the factorization of the spot-level expression matrix. The marker gene selection step is both a strength and a limitation. When marker genes are well characterized and specific to each cell type, SPOTlight can produce accurate proportion estimates with moderate computational cost. However, when marker genes are poorly defined or when cell types share many expressed genes, the method may struggle to distinguish between similar cell types. SPOTlight was not included in the cardiovascular and kidney disease benchmark, so direct performance comparisons with Cell2location and RCTD on those tissues are not available from that study. The method remains widely used in the spatial transcriptomics community, and its performance depends heavily on the quality of the marker gene selection. ## Practical Workflow for Deconvolution Analysis The deconvolution workflow involves several stages, from data preparation to quality assessment. Each stage requires specific decisions that affect the final output. ### Step 1: Prepare the Single-Cell Reference The reference dataset must contain the cell types expected in the spatial tissue. For tissues with known cellular composition, such as brain or heart, published reference atlases may be available. For less characterized tissues, researchers may need to generate their own single-cell or single-nucleus RNA-seq data. Quality control of the reference is essential. The reference should have sufficient cells per cell type, adequate sequencing depth, and minimal batch effects. Single-nucleus RNA-seq references are often preferred for spatial deconvolution because they avoid the dissociation artifacts that can affect single-cell RNA-seq, particularly for tissues such as brain and heart where enzymatic dissociation alters gene expression. The reference should be annotated to the cell-type level that matches the biological question. If the question concerns major cell types such as neurons, astrocytes, and oligodendrocytes, a coarse annotation is sufficient. If the question concerns subtypes such as excitatory neuron subtypes, the reference must be annotated at that finer resolution. ### Step 2: Process the Spatial Data Spatial transcriptomics data require standard preprocessing before deconvolution. This includes quality control to remove low-quality spots, normalization to account for differences in sequencing depth, and filtering to remove genes with low expression across all spots. The choice of normalization method can affect deconvolution results. Most deconvolution methods expect log-normalized or raw count data, and the method documentation should be consulted for the expected input format. Using the wrong input format is a common source of errors. ### Step 3: Select the Deconvolution Method The method selection depends on the tissue type, the reference data quality, and the computational resources available. The benchmark evidence suggests that Cell2location and RCTD are strong choices across diverse tissues, with Cell2location achieving the highest average performance in cardiovascular and kidney disease samples and RCTD achieving the best accuracy in cardiovascular samples specifically. For researchers with GPU access and a need for uncertainty estimates, Cell2location is a reasonable default choice. For researchers with limited computational resources or who need fast results, RCTD offers a simpler workflow with comparable accuracy for verifiable cell types. SPOTlight is appropriate when marker genes are well characterized and when the researcher has experience with non-negative matrix factorization approaches. ### Step 4: Run the Deconvolution and Assess Convergence Each method has specific convergence criteria that should be checked before interpreting results. Cell2location requires monitoring of the evidence lower bound to ensure the model has converged. RCTD requires checking that the optimization has reached a stable solution. SPOTlight requires verifying that the factorization has converged to a non-negative solution. Failure to check convergence is a common error. A model that has not converged produces proportion estimates that are not reliable, and downstream analyses based on these estimates will be misleading. ### Step 5: Validate the Deconvolution Results Validation is a critical step that is often skipped. The deconvolution results should be compared with known tissue anatomy, histological annotations, or independent measurements. For example, in brain tissue, expected cell-type proportions can be compared with known regional differences. In tumor tissue, immune cell proportions can be compared with pathological assessment. The benchmark study on cardiovascular and kidney disease samples used expert annotation to evaluate deconvolution performance. This approach, where an expert pathologist or histologist annotates regions of known cell-type composition, provides a ground truth for validation. Researchers should consider whether such validation is feasible for their tissue of interest. ## Options and Tradeoffs in Method Selection The choice among Cell2location, RCTD, and SPOTlight involves tradeoffs across several dimensions. Understanding these tradeoffs helps researchers make informed decisions based on their specific circumstances. ### Accuracy Across Tissue Types The large benchmarking study found that deconvolution performance varied substantially based on tissue type. Cell2location, RCTD, and SONAR emerged as top-performing methods across tissue types, but no single method was best for all tissues. This finding has practical implications: a method that performs well on brain tissue may not perform as well on cancer tissue, and vice versa. The cardiovascular and kidney disease benchmark provides tissue-specific evidence. All three methods tested, Cell2location, RCTD, and spatialDWLS, performed comparably well in deconvoluting verifiable cell types including smooth muscle cells and macrophages in vascular samples and podocytes in kidney samples. RCTD showed the best performance accuracy scores in cardiovascular disease samples, while Cell2location achieved the highest average performance across all test experiments. For researchers working on tissues not covered by these benchmarks, the recommendation is to validate deconvolution results against known biology or expert annotation. This validation is the only way to determine whether a specific method performs adequately for a specific tissue. ### Computational Resource Requirements Computational requirements differ substantially among the three methods. RCTD has the fastest computational time and the simplest workflow, requiring fewer computational dependencies. This makes RCTD suitable for researchers without access to high-performance computing clusters or GPUs. Cell2location requires higher computational intensity, and for large datasets, a GPU is often necessary. The Bayesian inference procedure involves iterative optimization that can be slow on CPU-only systems. Researchers planning to use Cell2location should budget for longer runtime and consider whether GPU resources are available. SPOTlight has moderate computational requirements. The non-negative matrix factorization is less computationally intensive than Bayesian inference but more intensive than the penalized least squares approach used by RCTD. ### Reference Data Requirements The amount of reference data needed for stable results differs among methods. The benchmark study found that Cell2location needed less reference data to converge, while RCTD required more reference data. This difference is practically important for researchers working with limited reference data, such as those studying rare or difficult-to-isolate cell types. For RCTD, the requirement for more reference data means that each cell type should be represented by a sufficient number of cells in the reference. When a cell type has few cells in the reference, the expression profile estimate is noisy, and the deconvolution results for that cell type are unreliable. Cell2location's ability to converge with less reference data is an advantage in data-limited settings, but this advantage comes at the cost of higher computational intensity. Researchers must weigh the tradeoff between reference data requirements and computational resources. ### Handling of Technical Variation Technical differences between single-cell RNA-seq and spatial transcriptomics data are a major source of deconvolution error. The single-cell reference is generated using droplet-based platforms with different capture efficiencies and sequencing depths compared with spatial platforms. These differences must be accounted for to avoid biased proportion estimates. Cell2location explicitly models platform-specific effects within its Bayesian framework. This modeling allows the method to learn and correct for technical differences between the reference and the spatial data. The result is proportion estimates that are less affected by platform-specific biases. RCTD also accounts for technical variation through its robust decomposition approach, but the mechanism differs from Cell2location. The penalized least squares approach is designed to be robust to outliers and technical noise, but it does not explicitly model platform effects in the same way as Cell2location. SPOTlight's handling of technical variation depends on the marker gene selection. If marker genes are chosen to be robust to platform differences, the method can produce reliable estimates. However, if marker genes are affected by platform-specific biases, the deconvolution results will reflect those biases. ## Observations and Measurements for Quality Assessment Quality assessment of deconvolution results requires specific observations and measurements. These assessments help researchers identify problems before they propagate to downstream analyses. ### Proportion Sum Checks The estimated cell-type proportions at each spot should sum to approximately one. Large deviations from this expectation indicate problems with the deconvolution. Some methods output proportions that are normalized to sum to one, while others output raw estimates that require normalization. Researchers should check the distribution of proportion sums across all spots. Spots with proportion sums substantially below one may have poor capture quality or may contain cell types not represented in the reference. Spots with proportion sums substantially above one indicate estimation errors. ### Cell-Type Proportion Distributions The distribution of estimated proportions for each cell type should be examined. Cell types that are known to be abundant in the tissue should show high proportions in many spots. Cell types that are known to be rare should show low proportions in most spots and higher proportions in specific regions. Unexpected proportion distributions can indicate problems. For example, if a cell type known to be rare shows high proportions across all spots, the reference may be misannotated, or the deconvolution method may be conflating that cell type with another. ### Spatial Pattern Assessment Deconvolution results should show coherent spatial patterns. Cell types that are known to be localized to specific tissue regions should show high proportions in those regions. Random or noisy spatial patterns suggest that the deconvolution is not capturing true biological variation. Visualizing the spatial distribution of each cell type's proportion estimates is a useful quality check. This visualization can be done using the spatial coordinates of the spots and coloring each spot by the estimated proportion of a specific cell type. ### Comparison with Histological Annotations When histological annotations are available, they provide a powerful validation tool. The benchmark study on cardiovascular and kidney disease samples used expert annotation to evaluate deconvolution performance. Researchers should consider whether similar validation is possible for their tissue. Histological comparison can identify systematic biases in deconvolution results. For example, if the deconvolution consistently overestimates immune cell proportions in regions that are histologically confirmed to have few immune cells, the reference or the method may need adjustment. ## Records and Documentation for Reproducibility Reproducibility is a core requirement for computational analyses. Deconvolution analyses should be documented with sufficient detail that another researcher can reproduce the results. ### Reference Dataset Documentation The reference dataset should be documented with its source, version, and processing steps. This documentation should include the number of cells, the number of cell types, the annotation scheme, and the quality control thresholds applied. If a published reference atlas is used, the version and download date should be recorded. The choice of reference dataset is a major determinant of deconvolution results. Different references for the same tissue can produce different proportion estimates, and this variability should be acknowledged in the analysis documentation. ### Method Parameters and Versions The deconvolution method version and all parameters should be recorded. This includes the method version, the random seed if applicable, the number of iterations, the convergence thresholds, and any method-specific parameters such as the number of marker genes for SPOTlight. Method versions change over time, and different versions may produce different results. Recording the exact version used in the analysis is essential for reproducibility. ### Computational Environment The computational environment should be documented, including the operating system, the software versions, and the hardware used. This documentation is particularly important for methods like Cell2location that may produce different results on different hardware due to differences in floating-point arithmetic or random number generation. ### Output Storage The deconvolution output should be stored in a format that preserves all relevant information. This includes the proportion estimates, the uncertainty estimates if available, and the convergence diagnostics. The output should be linked to the spatial coordinates of the spots so that downstream analyses can use the deconvolution results. ## Common Failure Patterns and Troubleshooting Several failure patterns recur in deconvolution analyses. Recognizing these patterns helps researchers diagnose problems and take corrective action. ### Reference Mismatch The most common failure pattern is a mismatch between the reference dataset and the spatial tissue. This mismatch can occur when the reference is from a different tissue state, a different developmental stage, or a different species. The deconvolution results will be misleading because the reference does not contain the cell types present in the spatial tissue. Troubleshooting this failure requires comparing the expression profiles of the reference and the spatial data. If the reference lacks genes that are highly expressed in the spatial data, or if the reference contains genes that are not detected in the spatial data, a reference mismatch is likely. ### Insufficient Reference Cells When a cell type has too few cells in the reference, the deconvolution results for that cell type are unreliable. This problem is particularly acute for rare cell types that are difficult to capture in single-cell RNA-seq experiments. The benchmark study found that RCTD required more reference data to converge, while Cell2location needed less. Researchers using RCTD should ensure that each cell type has a sufficient number of cells in the reference. A common threshold is at least 50 cells per cell type, but the appropriate threshold depends on the method and the tissue. ### Batch Effects in the Reference Batch effects in the reference dataset can introduce systematic biases into deconvolution results. If the reference contains cells from multiple batches with different technical characteristics, the deconvolution may attribute batch-specific expression differences to cell-type differences. Batch effect correction should be applied to the reference before deconvolution. Several tools are available for this purpose, and the choice of tool depends on the reference dataset structure. ### Platform-Specific Biases Technical differences between the single-cell reference platform and the spatial platform can bias deconvolution results. These biases include differences in capture efficiency, gene detection sensitivity, and sequencing depth. Cell2location explicitly models platform-specific effects, which makes it more robust to these biases. RCTD and SPOTlight are less explicit in their handling of platform differences, and researchers using these methods should be aware of this limitation. ### Convergence Failure Deconvolution methods that use iterative optimization can fail to converge. This failure produces proportion estimates that are not reliable, and the results should not be interpreted. Convergence diagnostics should be checked for every deconvolution run. For Cell2location, the evidence lower bound should be monitored. For RCTD, the optimization should reach a stable solution. For SPOTlight, the factorization should converge to a non-negative solution. ## Limitations of Deconvolution Methods Deconvolution methods have inherent limitations that researchers should understand before interpreting results. ### Inability to Resolve Similar Cell Types Deconvolution methods struggle to distinguish between cell types with similar expression profiles. This limitation is particularly relevant for subtypes of the same cell type, such as T-cell subtypes or neuronal subtypes. When cell types share many expressed genes, the deconvolution may assign proportions incorrectly between the similar types. The benchmark study on cardiovascular and kidney disease samples found that all three methods performed comparably well in deconvoluting verifiable cell types, including smooth muscle cells and macrophages in vascular samples and podocytes in kidney samples. These cell types have distinct expression profiles that are relatively easy to distinguish. For more similar cell types, performance would likely be lower. ### Dependence on Reference Quality All reference-based deconvolution methods depend on the quality of the reference dataset. A reference that is misannotated, contains batch effects, or lacks relevant cell types will produce misleading results. The adage garbage in, garbage out applies directly to deconvolution. Reference-independent methods exist, such as STEA, which does not require single-cell RNA-seq datasets as reference. These methods offer flexibility and computational efficiency, but they are not the focus of this comparison. Researchers who lack a suitable reference may consider reference-independent approaches. ### Aggregation at Spot Level Deconvolution estimates cell-type proportions at the spot level, not at the single-cell level. The output is a proportion estimate for each cell type at each spot, not the spatial location of individual cells. This distinction is important for interpreting results: deconvolution does not reveal the exact spatial arrangement of cells within a spot. Newer methods aim to achieve single-cell resolution from spot-based data. SpatialCell AI, for example, combines training-free operation with reference-free expression integration to produce per-cell output granularity. These methods are an active area of development, but they are not yet standard practice. ### Inability to Detect Unknown Cell Types Reference-based deconvolution can only estimate proportions of cell types present in the reference. If the spatial tissue contains a cell type that is not in the reference, the deconvolution will not detect it. Instead, the proportions of known cell types will be adjusted to account for the unknown cell type's contribution to the spot-level expression. This limitation is particularly relevant for disease tissues, which may contain cell types not present in healthy reference atlases. Researchers studying disease tissues should consider whether their reference adequately represents the cell types present in the diseased state. ## Safety and Regulatory Context for Research Applications Deconvolution methods are computational tools used in research settings. They are not regulated medical devices, and they do not have specific safety requirements. However, researchers should be aware of the regulatory context for spatial transcriptomics research, particularly when the research involves human tissue samples. ### Data Privacy and Consent Spatial transcriptomics data derived from human tissue samples are subject to data privacy regulations. Researchers must ensure that tissue samples were collected with appropriate informed consent and that the resulting data are handled in compliance with applicable regulations. The [NCBI](https://www.ncbi.nlm.nih.gov/) provides data resources and search systems that support responsible data sharing and access. ### Reproducibility Standards Funding agencies and journals increasingly require reproducible computational analyses. The [Galaxy Training Network](https://training.galaxyproject.org/) provides accessible workflow training and analysis tutorials that support reproducible research practices. The [nf-core documentation](https://nf-co.re/docs) describes community pipeline standards for reproducible workflow configuration. Researchers should follow these standards to ensure their deconvolution analyses are reproducible. ### Training and Education Researchers new to spatial transcriptomics analysis should seek appropriate training. The [EMBL-EBI Training](https://www.ebi.ac.uk/training) program offers bioinformatics learning pathways and data-resource training. The [Carpentries lessons](https://carpentries.org/lessons) provide foundational computing, data, shell, Git, and programming training that is useful for computational analysis. [Bioconductor](https://bioconductor.org/) provides official package, workflow, installation, and reproducible genomic-analysis documentation that is relevant to spatial transcriptomics analysis. ## Professional Escalation Criteria Researchers should seek expert assistance when deconvolution results are critical to a study and the standard troubleshooting steps do not resolve problems. The following situations warrant escalation to a bioinformatics specialist or a statistician with spatial transcriptomics expertise. ### Persistent Convergence Failure If a deconvolution method repeatedly fails to converge despite parameter adjustments, the problem may be with the reference dataset or the spatial data quality. A specialist can diagnose whether the issue is due to reference mismatch, batch effects, or data quality problems. ### Unexpected Cell-Type Proportions If the deconvolution produces cell-type proportions that contradict known tissue biology, the results should not be interpreted without further investigation. A specialist can help determine whether the reference is appropriate, whether the method is suitable for the tissue, or whether the spatial data have quality issues. ### Inconsistent Results Across Methods If different deconvolution methods produce substantially different results for the same dataset, the discrepancy should be investigated before proceeding. A specialist can help determine which method is more reliable for the specific tissue and data characteristics. ### Critical Downstream Decisions If deconvolution results will be used to make critical decisions, such as selecting regions for further analysis or drawing conclusions about disease mechanisms, expert review is warranted. The benchmark evidence shows that deconvolution performance varies substantially based on tissue type, and a specialist can help interpret results in the context of these limitations. ## A Practical Decision Framework for Method Selection Based on Data Characteristics The choice between Cell2location, RCTD, and SPOTlight often comes down to specific data characteristics that researchers can assess before running any deconvolution. A structured decision framework helps match the method to the data instead of relying on default preferences. The benchmarking evidence shows that deconvolution performance varied substantially based on tissue type, with Cell2location, RCTD, and SONAR emerging as top performers across diverse tissues while no single method dominated all scenarios. This section provides a practical framework for method selection based on observable data properties, reference quality, and biological context. ### Step 1: Assess Reference Dataset Completeness The reference dataset is the foundation of all three methods. Before selecting a deconvolution tool, researchers should evaluate whether the reference contains all expected cell types and whether each cell type has sufficient representation. The cardiovascular and kidney disease benchmark found that Cell2location needed less reference data to converge, while RCTD required more reference data for stable results. This difference has direct practical implications for method selection. For reference datasets where some cell types have fewer than 50 cells, Cell2location is the safer choice because its Bayesian framework can converge with smaller reference populations. For reference datasets with abundant cells across all expected cell types, RCTD becomes viable and offers faster runtime. SPOTlight requires well-characterized marker genes, so the reference must be annotated at a resolution that supports marker identification for each cell type of interest. Researchers should also assess whether the reference matches the tissue state of the spatial data. A reference from healthy tissue may not adequately represent disease tissue, which can contain activated or infiltrating cell types absent from the healthy reference. The benchmarking study across brain, cancer, and organ tissues demonstrated that performance varied by tissue type, suggesting that reference-tissue matching is a critical factor in method performance. ### Step 2: Evaluate Computational Resources and Time Constraints Computational resource availability often determines which method is practical for a given project. RCTD has the fastest computational time and the simplest workflow, requiring fewer computational dependencies. This makes RCTD the appropriate choice when results are needed quickly or when computing infrastructure is limited. Cell2location requires higher computational intensity, and for large spatial datasets, a GPU is often necessary to achieve reasonable runtime. Researchers without GPU access should estimate whether CPU-only processing time is acceptable for their dataset size. The Bayesian inference procedure involves iterative optimization that can be substantially slower than the penalized least squares approach used by RCTD. SPOTlight has moderate computational requirements. The non-negative matrix factorization approach is less computationally intensive than Bayesian inference but more intensive than RCTD. For researchers with moderate computing resources and well-characterized marker genes, SPOTlight offers a balanced option. ### Step 3: Consider the Biological Question and Required Resolution The biological question determines the required resolution of cell-type annotation. If the question concerns major cell types such as smooth muscle cells, macrophages, and podocytes, all three methods can perform comparably well, as demonstrated in the cardiovascular and kidney disease benchmark. These cell types have distinct expression profiles that are relatively easy to distinguish. If the question concerns closely related subtypes, such as T-cell subtypes or neuronal subtypes, the methods may struggle. Deconvolution methods have difficulty distinguishing between cell types with similar expression profiles. In these cases, the reference annotation must be carefully curated to define subtypes with sufficiently distinct expression signatures, and researchers should validate results against known biology. For questions that require single-cell resolution instead of spot-level proportions, none of the three methods is sufficient. Newer approaches such as SpatialCell AI achieve per-cell output granularity from spot-based data, but these methods are not yet standard practice. Researchers should recognize the limitation of spot-level deconvolution and consider whether the biological question requires finer resolution. ### Step 4: Match Method to Tissue Type and Platform The benchmarking evidence indicates that deconvolution performance varies by tissue type. The large benchmarking study across brain, cancer, and organ tissues found that Cell2location, RCTD, and SONAR were top performers, but performance varied substantially based on tissue type. The cardiovascular and kidney disease benchmark found that RCTD showed the best performance accuracy in cardiovascular disease samples, while Cell2location achieved the highest average performance across all test experiments. For researchers working on tissues not covered by these benchmarks, the recommendation is to validate deconvolution results against known biology or expert annotation. The benchmark study used expert annotation to evaluate deconvolution performance, and this approach provides a ground truth for validation. Researchers should consider whether such validation is feasible for their tissue of interest. Platform differences also matter. Sequencing-based platforms such as Visium and Stereo-seq capture multiple cells per spot, while imaging-based platforms such as MERFISH, Xenium, and CosMx provide subcellular resolution. The deconvolution problem is specific to sequencing-based platforms, and the choice of method should account for the platform used to generate the spatial data. ### Step 5: Run a Pilot Analysis with Multiple Methods When the optimal method is unclear, running a pilot analysis with two or three methods on a subset of the data provides empirical evidence for method selection. The pilot should use a representative subset of spots and compare the proportion estimates across methods. If the methods produce similar results, the choice can be based on computational resources and workflow simplicity. If the methods produce substantially different results, the discrepancy should be investigated before proceeding. The pilot analysis should include convergence checks for each method. A method that fails to converge on the pilot data will likely fail on the full dataset. The pilot also provides an opportunity to assess runtime and resource requirements before committing to a full analysis. ### Step 6: Document the Decision Rationale The method selection rationale should be documented for reproducibility. This documentation should include the reference dataset characteristics, the computational resources available, the biological question, and the results of any pilot analyses. The documentation should also record the method version and all parameters used. Reproducibility standards from the [nf-core documentation](https://nf-co.re/docs) describe community pipeline standards for workflow configuration. The [Galaxy Training Network](https://training.galaxyproject.org/) provides accessible workflow training and analysis tutorials that support reproducible research practices. Following these standards ensures that the method selection and deconvolution analysis can be reproduced by other researchers. ### Decision Matrix for Common Scenarios The following decision matrix summarizes the recommended method for common research scenarios based on the benchmark evidence and practical considerations. | Scenario | Recommended Method | Rationale | | --- | --- | --- | | Limited reference data, GPU available | Cell2location | Converges with less reference data, models platform effects | | Limited reference data, no GPU | SPOTlight | Moderate computational requirements, works with smaller references if markers are defined | | Abundant reference data, fast results needed | RCTD | Fastest runtime, simplest workflow, comparable accuracy for verifiable cell types | | Cardiovascular disease samples | RCTD | Best performance accuracy in cardiovascular disease benchmark | | Diverse tissues, average performance priority | Cell2location | Highest average performance across cardiovascular and kidney disease test experiments | | Well-characterized marker genes available | SPOTlight | Marker-guided factorization produces accurate results with moderate computation | | Unknown tissue composition | Validate with expert annotation | No method is reliable without validation against known biology | ### Limitations of the Decision Framework This decision framework is based on the available benchmark evidence, which covers specific tissues and platforms. The large benchmarking study evaluated 21 deconvolution methods across 37 datasets spanning brain, cancer, and organ tissues and four distinct spatial technologies. The cardiovascular and kidney disease benchmark evaluated three methods on human data from patients in different pathological states. The framework does not account for all possible scenarios. Researchers working on tissues not covered by these benchmarks should treat the recommendations as starting points instead of definitive answers. The only way to determine whether a specific method performs adequately for a specific tissue is to validate the deconvolution results against known biology or expert annotation. The framework also assumes that the reference dataset is of adequate quality. A reference that is misannotated, contains batch effects, or lacks relevant cell types will produce misleading results regardless of the method chosen. Reference quality assessment should be performed before method selection, and reference-independent methods such as STEA should be considered when a suitable reference is unavailable. ### Integration with Existing Workflows The decision framework integrates with existing spatial transcriptomics analysis workflows. The [Bioconductor](https://bioconductor.org/) project provides official package, workflow, installation, and reproducible genomic-analysis documentation that is relevant to spatial transcriptomics analysis. The [EMBL-EBI Training](https://www.ebi.ac.uk/training) program offers bioinformatics learning pathways and data-resource training that can help researchers develop the skills needed to implement this framework. The [Carpentries lessons](https://carpentries.org/lessons) provide foundational computing, data, shell, Git, and programming training that is useful for implementing reproducible deconvolution workflows. The [NCBI](https://www.ncbi.nlm.nih.gov/) provides data resources and search systems that support reference dataset selection and quality assessment. Researchers should document the decision framework application in their analysis records. This documentation should include the data characteristics assessed at each step, the rationale for method selection, and the results of any pilot analyses. This documentation supports reproducibility and provides a basis for troubleshooting if the deconvolution results are later found to be problematic. ## Frequently Asked Questions ### What is the difference between single-cell RNA-seq and single-nucleus RNA-seq for deconvolution references? Single-cell RNA-seq captures whole cells, while single-nucleus RNA-seq captures only nuclei. For tissues where enzymatic dissociation alters gene expression, such as brain and heart, single-nucleus RNA-seq is often preferred because it avoids dissociation artifacts. The choice affects the reference expression profiles and therefore the deconvolution results. Researchers should match the reference type to the biological question and the tissue characteristics. ### How many cells per cell type are needed in the reference for reliable deconvolution? The required number depends on the method and the cell type. The benchmark study found that Cell2location needed less reference data to converge, while RCTD required more reference data. A common practice is to ensure at least 50 cells per cell type, but the appropriate threshold depends on the method, the cell type, and the tissue. Researchers should assess the stability of deconvolution results as the reference size varies. ### Can deconvolution methods distinguish between closely related cell types? Deconvolution methods struggle to distinguish between cell types with similar expression profiles. The benchmark study found that all three methods performed comparably well for verifiable cell types with distinct expression profiles, such as smooth muscle cells and macrophages. For closely related subtypes, such as T-cell subtypes, performance is likely lower. Researchers should consider whether the reference annotation matches the resolution needed for their biological question. ### What is the role of marker genes in SPOTlight deconvolution? SPOTlight uses marker genes to guide the non-negative matrix factorization. The marker genes are identified from the reference dataset and should be specific to each cell type. The quality of the marker gene selection directly affects the deconvolution accuracy. When marker genes are well characterized and specific, SPOTlight can produce accurate results. When marker genes are poorly defined, the method may struggle to distinguish between cell types. ### How should deconvolution results be validated? Deconvolution results should be validated against known tissue biology, histological annotations, or independent measurements. The benchmark study on cardiovascular and kidney disease samples used expert annotation to evaluate deconvolution performance. Researchers should compare deconvolution results with expected cell-type proportions in known tissue regions and assess whether the spatial patterns are biologically coherent. ### What computational resources are needed for Cell2location? Cell2location requires higher computational intensity than RCTD or SPOTlight. For large spatial datasets, a GPU is often necessary to achieve reasonable runtime. The Bayesian inference procedure involves iterative optimization that can be slow on CPU-only systems. Researchers planning to use Cell2location should budget for longer runtime and consider whether GPU resources are available. ### What should be done if different deconvolution methods produce different results? Different deconvolution methods can produce different results for the same dataset due to differences in their statistical models and assumptions. The benchmark evidence shows that performance varies by tissue type and method. Researchers should investigate the discrepancies, assess which method is more reliable for their specific tissue and data characteristics, and consider validating the results against known biology or expert annotation. ### Are there reference-independent alternatives to Cell2location, RCTD, and SPOTlight? Yes, reference-independent methods exist. STEA is a reference-independent enrichment-based annotation algorithm that does not require single-cell RNA-seq datasets as reference. SpatialCell AI combines training-free operation with reference-free expression integration to achieve single-cell resolution from spot-based data. These methods offer flexibility when a suitable reference is unavailable, but they are not yet standard practice and should be evaluated for the specific research question. ## Related Bioinformatics Guides - [Spatial Transcriptomics Integration: Methods for Combining Data Across Platforms](/knowledge/bioinformatics/spatial-transcriptomics-integration-methods-for-combining-data-across-platforms) - [Spatial Transcriptomics vs. Single-Cell RNA Sequencing: Which Approach Fits Your Research?](/knowledge/bioinformatics/spatial-transcriptomics-vs-single-cell-rna-sequencing-which-approach-fits-your-research) - [Spatial Transcriptomics Methods: A Guide to Experimental Approaches](/knowledge/bioinformatics/spatial-transcriptomics-methods-a-guide-to-experimental-approaches) - [Spatial Transcriptomics Differential Expression: Methods and Best Practices](/knowledge/bioinformatics/spatial-transcriptomics-differential-expression-methods-and-best-practices) - [Spatial Transcriptomics Data Analysis: A Guide to Preprocessing, Integration, and Interpretation](/knowledge/bioinformatics/spatial-transcriptomics-data-analysis-a-guide-to-preprocessing-integration-and-interpretation) ## Related Clinical & Scientific Guides * [A Practical Guide to Detecting Antimicrobial Resistance Genes in Shotgun Metagenomic Data](/knowledge/bioinformatics/a-practical-guide-to-detecting-antimicrobial-resistance-genes-in-shotgun-metagenomic-data) * [Computational Immunology: Modeling the Immune System](/knowledge/bioinformatics/computational-immunology-modeling-the-immune-system) * [How to Set Hard Filters for Germline Variant Calling: A Practical Guide to GATK Best Practices](/knowledge/bioinformatics/how-to-set-hard-filters-for-germline-variant-calling-a-practical-guide-to-gatk-best-practices) ## References and Further Reading - [NCBI Data Resources](https://www.ncbi.nlm.nih.gov/). National Center for Biotechnology Information. - [EMBL-EBI Training](https://www.ebi.ac.uk/training). European Bioinformatics Institute. - [Bioconductor](https://bioconductor.org/). Bioconductor Project. - [Galaxy Training Network](https://training.galaxyproject.org/). Galaxy Project. - [nf-core Documentation](https://nf-co.re/docs). nf-core. - [The Carpentries Lessons](https://carpentries.org/lessons). The Carpentries. - [A Comprehensive Benchmarking of Spatial Deconvolution and Domain Detection Methods across Diverse Tissues and Spatial Transcriptomic Technologies](https://doi.org/10.21203/rs.3.rs-9676637/v1). 2026. - [Decoding cardiac homeostasis and injury: the evolving landscape of spatial transcriptomics.](https://doi.org/10.3389/fcell.2026.1772507). 2026. - [SpatialCell AI achieves reference-free single-cell resolution from spot-based spatial transcriptomics through morphology-guided enhancement.](https://doi.org/10.1038/s41598-026-55246-w). 2026. - [STEA: Histologically Validated and Reference-Independent Major Cell-Type Annotation for Spatial Transcriptomics Reveals Relevant Cellular Organization and Architecture of Tumor Microenvironment.](https://doi.org/10.3390/cancers18091425). 2026. - [HistoMap: Reconstructing Spatially Resolved Single-Cell Profiles from Bulk RNA-Seq to Decipher the Immune-Excluded Microenvironment in Colon Cancer](https://europepmc.org/article/PMC/PMC13300051). 2026. - [Paired comparison of tumor core and airway lumen (BALF) microbiomes in lung adenocarcinoma: deciphering specific Bacillus enrichment and immunomodulation](https://doi.org/10.3389/fcimb.2026.1768287). Frontiers in Cellular and Infection Microbiology, 2026. - [Cell-type deconvolution methods for spatial transcriptomics](https://doi.org/10.1038/s41576-025-00845-y). Nature reviews genetics, 2025. - [From pixels to cell types: a comprehensive review of computational methods for spatial transcriptomics deconvolution](https://doi.org/10.1186/s44342-025-00055-2). Genomics & Informatics, 2025. - [STDSN: Domain Separation Network for Transfer Learning in Spatial Transcriptomics Deconvolution](https://doi.org/10.1109/BIBM66473.2025.11357033). IEEE International Conference on Bioinformatics and Biomedicine, 2025. - [A systematic evaluation of state-of-the-art deconvolution methods in spatial transcriptomics: insights from cardiovascular disease and chronic kidney disease](https://doi.org/10.3389/fbinf.2024.1352594). Frontiers Bioinform., 2024. - [All You Need is Color: Image Based Spatial Gene Expression Prediction Using Neural Stain Learning](https://doi.org/10.1007/978-3-030-93733-1_32). Communications in Computer and Information Science, 2021. > This article is educational and does not replace validated analysis plans, institutional policy, clinical interpretation, or specialist review.