Single-Cell RNA-seq Clustering and Cell-Type Annotation Pipelines

Introduction

Single-cell RNA sequencing (scRNA-seq) has transformed the study of cellular heterogeneity in tissues, enabling the identification of distinct cell types and states based on transcriptomic profiles [1, 2]. The core analytical challenge in scRNA-seq is the accurate clustering of cells into biologically meaningful groups and the subsequent annotation of those groups with cell-type labels [3]. This process is fundamental to downstream analyses such as differential expression, trajectory inference, and spatial mapping [4, 5]. In veterinary medicine, scRNA-seq clustering pipelines are increasingly applied to understand host-pathogen interactions, tissue development, and disease pathogenesis in non-human species [6]. The computational workflow typically involves quality control (QC), normalization, feature selection, dimensionality reduction, clustering, and annotation [2, 7]. Each step requires careful methodological choices that can profoundly affect biological conclusions [8, 7]. This article provides an exhaustive technical review of clustering and annotation pipelines for scRNA-seq data, grounded in the published literature and with a focus on algorithmic principles and comparative biology.

Preprocessing and Quality Control

Raw scRNA-seq data consist of count matrices representing gene expression per cell [2]. Preprocessing begins with QC to remove low-quality cells, empty droplets, and doublets [2, 9]. Common metrics include the number of unique molecular identifiers (UMIs), number of genes detected, and the proportion of mitochondrial reads [2]. For single-nucleus RNA-seq (sNuc-RNAseq), intronic reads constitute a substantial fraction of mapped reads and can be leveraged to improve cell detection and clustering resolution [9]. The ABACUS UMI counter, for example, incorporates intronic reads to nearly double the number of nuclei identified compared to standard exonic-only pipelines [9]. Ambient RNA, originating from cell lysis prior to droplet partitioning, is a major confounder that can be estimated using variant-based methods such as souporcell [10, 11]. Souporcell clusters cells by genotype without requiring a reference, simultaneously identifying cross-genotype doublets and estimating ambient RNA contamination [10, 11]. Doublet detection is critical because doublets can form clusters with intermediate transcriptional profiles that obscure true cell types [10]. After QC, normalization adjusts for differences in sequencing depth and capture efficiency [2]. Log-normalization and scaling are standard, but more sophisticated methods such as scSVA use variational autoencoders to impute dropout events while avoiding bias [12]. Matrix completion with side information (gene-gene associations) has also been proposed to recover zero-inflated expression values [13].

Feature Selection and Dimensionality Reduction

Feature selection identifies a subset of highly variable genes (HVGs) that capture biological variation while reducing noise [14]. Conventional HVG selection based on variance is susceptible to inflated variances due to inconsistent transcriptomic sampling [14]. An alternative approach uses co-occurrence correlations between genes to select features, which improves clustering accuracy and separability across multiple datasets [14]. After feature selection, dimensionality reduction is performed, typically using principal component analysis (PCA) on the scaled expression matrix of HVGs [1, 7]. The number of principal components (PCs) is a key parameter; 30-50 PCs are commonly used [8, 7]. Non-linear methods such as t-SNE and UMAP are employed for visualization but are not recommended for clustering directly due to potential distortion of global structure [1]. The p-ClustVal method introduces a p-adic number theory-based transformation that enhances cluster discernibility in high-dimensional scRNA-seq data, improving performance of downstream clustering algorithms in 91% of tested cases [15]. This transformation operates without ground truth labels and uses a data-centric heuristic to determine optimal parameters [15].

Clustering Algorithms

Clustering is the central step in delineating putative cell types or states [1, 3]. Algorithms can be broadly categorized as classical (e.g., k-means), density-based (e.g., DBSCAN), graph-based (e.g., Louvain, Leiden), and ensemble methods [8, 16]. Graph-based clustering, particularly Louvain modularity optimization on a k-nearest neighbor graph, is widely used due to its scalability and performance [7, 16]. A systematic benchmark of classical, density-based, and ensemble clustering for scRNA-seq data found that a hybrid workflow combining k-means, DBSCAN, and bootstrap consensus (25-fold) achieved an adjusted Rand index of 0.79 on a human pancreas dataset, outperforming raw Louvain and deep-learning baselines [8]. The SC3s algorithm provides efficient scaling of consensus clustering to millions of cells by using a k-means-based approach that scales linearly with cell number [16]. For datasets containing multiple genotypes (e.g., multiplexed samples), souporcell clusters cells by genetic variants detected in scRNA-seq reads, enabling donor assignment and doublet removal without reference genotypes [10, 11]. Deep learning methods have also been applied: the GAEKLRR method uses a graph autoencoder to learn a low-rank representation, followed by relaxed k-means for soft clustering, which is robust to noise and outliers [17]. The SnapCCESS framework integrates multimodal single-cell omics data (e.g., gene expression and chromatin accessibility) using an ensemble of variational autoencoder embeddings, generating consensus clusters that outperform single-modality approaches [18]. For single-cell multi-omics data, the GSTRPCA algorithm applies irregular tensor robust PCA to preserve data structure and extract hidden relationships, significantly improving clustering performance [19].

Cell-Type Annotation

After clustering, cell types must be assigned to each cluster [20, 3]. Annotation can be performed manually using known marker genes, automatically using supervised classifiers, or semi-automatically using reference-based mapping [20]. A comprehensive evaluation of 17 cell-based and 9 cluster-based labelling algorithms on cancer datasets found that cell-based methods (e.g., scPred, SVM) generally achieved higher accuracy and speed, while cluster-based methods performed better for non-malignant cell types [20]. The scClustViz tool provides an interactive interface for assessing clustering results using differential gene expression between clusters, helping to select appropriate resolution parameters and identify marker genes [3]. For functional annotation, the PAGER-scFGA pipeline integrates gene-set enrichment analysis with cell clustering to infer cell functions and molecular mechanisms along trajectories, using differentially expressed genes from pairwise cluster comparisons [5]. In spatial transcriptomics, the STIE algorithm aligns spot-level transcriptomes to nuclear morphology from histology images, achieving single-cell resolution deconvolution and clustering that improves concordance with scRNA-seq-derived signatures [21]. For model organisms like Drosophila, spatial mapping of scRNA-seq clusters can be validated using in situ hybridization data and existing anatomical atlases [4]. In veterinary contexts, annotation often relies on cross-species mapping of marker genes, which requires careful consideration of evolutionary conservation and tissue-specific expression [6].

Integration and Scalability

As scRNA-seq datasets grow to millions of cells, computational efficiency becomes critical [22, 16]. GPU-accelerated frameworks such as rapids-singlecell provide near drop-in replacements for CPU-based analysis steps, achieving speedups of up to several hundred-fold for preprocessing, dimensionality reduction, neighborhood graph construction, clustering, and batch correction [22]. This enables interactive exploration and iterative hypothesis testing on large datasets [22]. The BERLIN pipeline facilitates reprocessing of public scRNA-seq data by integrating data retrieval, QC, normalization, dimension reduction, clustering, and integration into a Shiny App for visualization [23]. For cohort studies, scACCorDiON uses optimal transport on directed weighted graphs to analyze cell-cell communication networks across patients, enabling clustering of samples based on communication patterns that correlate with disease status [24]. Batch correction is essential when integrating multiple samples or experiments; methods such as mutual nearest neighbors (MNN) and Harmony are commonly used, but the choice of correction method can affect clustering results [22, 23].

Evaluation and Benchmarking

Objective evaluation of clustering and annotation pipelines is challenging due to the lack of ground truth labels in most biological datasets [7]. Self-supervised benchmarking using count splitting and Poisson noise modeling enables validation without external labels; a benchmark of 120 pipelines identified that anti-correlation-based feature selection paired with locally weighted Louvain modularity on Euclidean distance of 50 PCs with cluster validation yielded the best performance [7]. The p-ClustVal method was evaluated across 30 experiments and over 1400 observations, demonstrating improvement in 91% of cases [15]. For annotation, the evaluation of 26 algorithms on cancer datasets provided guidance on algorithm selection based on performance metrics and computational cost [20]. The scClustViz tool uses differential expression between clusters as a metric for assessing biological relevance at multiple resolution levels [3]. The QCC (Quality Control of Cluster) and QCM (Quality Control of Model) metrics introduced with the sparsely-connected autoencoder (SCA) framework quantify the ability of neural networks to reconstruct valuable cell clusters [25].

FAQ

What is the first step in a scRNA-seq clustering pipeline?

The first step is quality control (QC) to filter low-quality cells, empty droplets, and doublets, followed by normalization to account for sequencing depth differences [2, 9].

How are highly variable genes selected for clustering?

Highly variable genes are typically selected based on variance across cells, but co-occurrence correlation-based selection can improve clustering accuracy by avoiding inflated variances due to sampling noise [14].

Which clustering algorithm is most commonly used for scRNA-seq?

Graph-based clustering using Louvain or Leiden modularity optimization on a k-nearest neighbor graph is most common due to its scalability and performance [7, 16].

How can cell types be annotated automatically?

Automatic annotation can be performed using supervised classifiers (e.g., scPred, SVM) trained on reference datasets, or cluster-based methods that assign labels based on marker gene enrichment [20].

What methods exist for doublet detection in scRNA-seq?

Doublet detection can be performed using transcriptional profile-based methods or genotype-based methods such as souporcell, which identifies cross-genotype doublets without a reference [10, 11].

How can large scRNA-seq datasets be analyzed efficiently?

GPU-accelerated frameworks like rapids-singlecell provide substantial speedups for preprocessing, dimensionality reduction, and clustering, enabling analysis of millions of cells in minutes [22].

What is the role of dimensionality reduction in clustering?

Dimensionality reduction (e.g., PCA) reduces noise and computational burden while preserving biological variation; the resulting low-dimensional embeddings are used for neighborhood graph construction and clustering [1, 7].

How are clustering results validated without ground truth?

Self-supervised methods using count splitting and Poisson noise modeling can validate clusters by assessing reproducibility and biological relevance [7]. Tools like scClustViz use differential expression between clusters to evaluate fit [3].

Can scRNA-seq clustering be applied to veterinary species?

Yes, scRNA-seq clustering pipelines are applicable to any species, though annotation may require cross-species mapping of marker genes and careful consideration of evolutionary conservation [6].

What is the difference between cell-based and cluster-based annotation methods?

Cell-based methods assign labels to individual cells using classifiers trained on reference data, while cluster-based methods assign labels to entire clusters based on aggregate expression profiles [20].

References

[1] Beccuti M, Calogero R. Single-Cell RNAseq Clustering. Methods Mol Biol. 2023. URL: https://www.semanticscholar.org/paper/237771e8970a43746e7d35b807f49f3472f46e26

[2] Olivero M, Calogero R. Single-Cell RNAseq Data QC and Preprocessing. Methods Mol Biol. 2023. URL: https://www.semanticscholar.org/paper/e8065c11977abcf96853aea473c73bbf58cc799a

[3] Innes BT, Bader GD. scClustViz – Single-cell RNAseq cluster assessment and visualization. F1000Res. 2018. URL: https://www.semanticscholar.org/paper/67a3ffc16a37c176cf85a8990078f1405ab87e54

[4] Mohr S, Tattikota SG, Xu J, et al. Methods and tools for spatial mapping of single-cell RNAseq clusters in Drosophila. Genetics. 2021. URL: https://www.semanticscholar.org/paper/7ea1bb06aeede1bc5b65f0426f15120d59d2d447

[5] Huang F, Welner RS, Chen J, et al. PAGER-scFGA: unveiling cell functions and molecular mechanisms in cell trajectories through single-cell functional genomics analysis. Front Bioinform. 2024. URL: https://www.semanticscholar.org/paper/fb25912c21daf6db0d0c18e15b0582a31c01d6a5

[6] Freeland T, Xie N, Rennhack A, et al. Single cell analysis of chronic myeloid leukemia bone marrow from time of TKI discontinuation reveals stem cell alterations associated with recurrence versus treatment free remission. Blood. 2025. URL: https://www.semanticscholar.org/paper/27e4f4b63d6a51952557787632928d48d5017848 *** Disclaimer: This article is for educational and informational purposes only. It is not intended to substitute for professional veterinary advice, diagnosis, treatment, or regulatory guidance. Always consult a licensed veterinarian or qualified specialist regarding animal health, disease diagnosis, and therapeutic decisions.

[7] Tyler SR, Schadt E, Guccione E. Self-supervised Benchmarking for scRNAseq Clustering. bioRxiv. 2023. URL: https://www.semanticscholar.org/paper/71cd6e65b2dc593d144859dd57de121494bfb537

[8] Aggarwal SK, Lal N, Sinha A. A Systematic Benchmark of Classical, Density-Based, and Ensemble Clustering for scRNA-Seq. ICRITO. 2025. URL: https://www.semanticscholar.org/paper/4b0d5a04e016221deb293b080a06a04571b4252a

[9] Xi S, Gibilisco L, Kummer MP, et al. ABACUS: A flexible UMI counter that leverages intronic reads for single-nucleus RNAseq analysis. bioRxiv. 2020. URL: https://www.semanticscholar.org/paper/303c6b439169956fbd54d0ab6fd1f51c7ef5f1df

[10] Heaton H, Talman A, Knights AJ, et al. souporcell: Robust clustering of single cell RNAseq by genotype without reference genotypes. Nat Methods. 2020. URL: https://www.semanticscholar.org/paper/4df0eab2c8ac71963850a7c4d2e1c71bdcb3752a

[11] Heaton H, Talman A, Knights AJ, et al. souporcell: Robust clustering of single cell RNAseq by genotype and ambient RNA inference without reference genotypes. bioRxiv. 2019. URL: https://www.semanticscholar.org/paper/51ae1ad83435847e55506060e07bd217c0cad6cb

[12] Sun S, Liu Y, Shang X. Deep generative autoencoder for low-dimensional embedding extraction from single-cell RNAseq data. IEEE Int Conf Bioinform Biomed. 2019. URL: https://www.semanticscholar.org/paper/88b71d54f28150a83e9f7584118ff24d54366b95

[13] Wang T, Nabavi S. Single-cell RNAseq Imputation Based on Matrix Completion with Side Information. IEEE Int Conf Bioinform Biomed. 2019. URL: https://www.semanticscholar.org/paper/f30e396bf3aa4ab5faab50ea8d69e087b61eb2e6

[14] Walsh ES, Ghashghaei HT, Peng X. Feature selection using co-occurrence correlation improves cell clustering and embedding in single cell RNAseq data. IEEE Int Conf Bioinform Biomed. 2021. URL: https://www.semanticscholar.org/paper/7988b55e53c1a9f7de0ecc59a3eca9f123b79efc

[15] Sharma P, Mishra SR, Kurban H, et al. p-ClustVal: A Novel p-adic Approach for Enhanced Clustering of High-Dimensional Single Cell RNASeq Data. bioRxiv. 2024. URL: https://www.semanticscholar.org/paper/fd535062441f1780c59205d10a8195550f21744d

[16] Quah FX, Hemberg M. SC3s - efficient scaling of single cell consensus clustering to millions of cells. bioRxiv. 2021. URL: https://www.semanticscholar.org/paper/c535e6b9357d4b456079c0047a7bd1f631f52251

[17] Wang LP, Shang J, Dai L, et al. GAEKLRR: A novel clustering method of the low-rank representation based on graph auto-encoder and relaxed k-means for single-cell type identification. IEEE Int Conf Bioinform Biomed. 2025. URL: https://www.semanticscholar.org/paper/f98ea7a147edaf0fe01662dd194702c92ab9a1c7

[18] Yu L, Liu CL, Yang JYH, et al. Ensemble deep learning of embeddings for clustering multimodal single-cell omics data. bioRxiv. 2023. URL: https://www.semanticscholar.org/paper/6fff73266dfc869ce31bc8cf6a6650a3345f6c7e

[19] Cui LB, Guo G, Ng MK, et al. GSTRPCA: irregular tensor singular value decomposition for single-cell multi-omics data clustering. Brief Bioinform. 2024. URL: https://www.semanticscholar.org/paper/840d8485a391811f3a68d7296db2dce8470e057a

[20] Christensen E, Luo P, Turinsky AL, et al. Evaluation of single-cell RNAseq labelling algorithms using cancer datasets. Brief Bioinform. 2022. URL: https://www.semanticscholar.org/paper/6a8a72e82851da0fa4d1e8284fa70f8a8183f21b

[21] Zhu S, Kubota N, Wang S, et al. Single-cell level deconvolution, convolution, and clustering in spatial transcriptomics by aligning spot level transcriptome to nuclear morphology. bioRxiv. 2023. URL: https://www.semanticscholar.org/paper/7ae87f0106a4c9e2ee5d0d7815e87a636ee28f2f

[22] Dicks S, Heumos L, May L, et al. GPU-accelerated single-cell analysis at scale with rapids-singlecell. J. 2026. URL: https://www.semanticscholar.org/paper/b856e4c5d15877811bd6627e6153e5020b0cfb59

[23] Nobles G, Obermayer AN, Yang C, et al. BERLIN: Basic Explorer for single-cell RNAseq analysis and cell Lineage Determination. bioRxiv. 2023. URL: https://www.semanticscholar.org/paper/0337201592a549c0d88ba16bb2b8e2f4ef421b83

[24] Nagai J, Schaub MT, Costa IG. scACCorDiON: a clustering approach for explainable patient level cell–cell communication graph analysis. bioRxiv. 2024. URL: https://www.semanticscholar.org/paper/de3b1a0e867007786509113510f32a5d3c95b240

[25] Alessandri L, Cordero F, Beccuti M, et al. Sparsely-connected autoencoder (SCA) for single cell RNAseq data mining. npj Syst Biol Appl. 2020. URL: https://www.semanticscholar.org/paper/4b36c2dac79de842e117954fa774946b9fc05fc7

[26] Bornet E, Westermann AJ. Single-bacterium RNA-seq protocol to uncover heterogeneous expression of coding and noncoding genes in Bacteroides thetaiotaomicron. STAR Protoc. 2026. URL: https://pubmed.ncbi.nlm.nih.gov/42391002/

[27] Zhen C, Wang Y, Han L, et al. A novel framework for single-cell Hi-C clustering based on graph-convolution-based imputation and two-phase-based feature extraction. bioRxiv. 2021. URL: https://www.semanticscholar.org/paper/4dd7e2d67925482324b90dd8a422949586d872ca