Foundation Models for Genomics: From Single Cells to Health Trajectories

Foundation models in genomics are large-scale machine learning systems pretrained on vast amounts of biological sequence data that can be adapted to many downstream tasks including single-cell RNA-seq analysis, variant effect prediction, and health trajectory modeling. For researchers and analysts entering this field, these models represent a shift from training task-specific models from scratch toward leveraging shared representations learned from unlabeled genomic data. This article explains what foundation models are, how they are being applied across genomics, which models are available, and what practical considerations matter when using them.

What Are Foundation Models in Genomics

A foundation model is a machine learning system trained on broad data at scale that can be adapted to a wide range of downstream tasks. In genomics, these models are typically transformer architectures pretrained on DNA sequences, RNA expression profiles, or other biological data. The pretraining phase learns general patterns in the data without requiring labeled examples. After pretraining, the model can be fine-tuned on specific tasks with relatively small amounts of labeled data.

The core idea draws from natural language processing where models like BERT and GPT learn language patterns from massive text corpora. Genomic foundation models apply the same principle to biological sequences. DNA sequences can be tokenized similarly to words, and the transformer architecture learns contextual relationships between sequence elements. The Nucleotide Transformer study demonstrated that models pretrained on DNA sequences from 3,202 human genomes and 850 genomes across diverse species produce context-specific representations that enable accurate predictions even in low-data settings.

The practical value for researchers is the ability to solve genomics problems without collecting and labeling massive training datasets for every new task. A model pretrained on genome sequences can be fine-tuned at relatively low cost for variant prioritization, gene expression prediction, or regulatory element identification. The Evo model trained on millions of prokaryotic and phage genomes showed zero-shot function prediction competitive with domain-specific language models and demonstrated the ability to generate functional CRISPR-Cas and transposon systems.

Training Data and Model Architectures

DNA Sequence Models

DNA foundation models differ primarily in their training data, architecture, and tokenization strategy. The Nucleotide Transformer family ranges from 50 million to 2.5 billion parameters and integrates information from thousands of human genomes plus genomes from hundreds of other species. These models learn to focus attention on key genomic elements without supervision, which improves variant prioritization.

The MutBERT model addresses a specific limitation of earlier approaches. Human population data such as the 1000 Genomes Project contains sparse and redundant SNP information. Standard masked language models trained directly on whole-genome sequences may struggle to learn these variations efficiently. MutBERT represents the genome as a probabilistic distribution over observed allele frequencies, focusing on informative genomic variations while maintaining computational efficiency. In evaluations against DNABERT-2 and various Nucleotide Transformer versions, MutBERT consistently ranked among the top-performing models across multiple downstream prediction tasks.

The Evo model uses a long-context architecture trained on prokaryotic and phage genomes. It generalizes across DNA, RNA, and proteins, enabling predictions at multiple scales from molecular function to whole-organism fitness effects of small mutations. Evo can also generate megabase-scale sequences with plausible genomic architecture.

Single-Cell Models

Single-cell foundation models operate on gene expression data instead of raw DNA sequence. These models are pretrained on large atlases of single-cell RNA-seq data and learn representations of cellular states. The review of plant single-cell genomics describes how transformer-based architectures including GPT and BERT variants are applied to cell-type annotation, gene network modeling, and multi-omics integration in plants. These models address challenges including data sparsity, dropout events, and class imbalance in single-cell datasets.

The SIGnature framework uses attribution scores derived from single-cell RNA-seq foundation models to score gene importance within cellular contexts. Attribution scores reduce technical noise, emphasize regulatory genes, and enable cross-dataset comparison. The developers demonstrated utility by searching 400 studies to identify associations between the MS1 monocyte signature and multiple hyperinflammatory conditions including Kawasaki disease, with experimental validation confirming that serum from affected individuals induces the MS1 phenotype.

Multimodal and Multiscale Approaches

Recent work has moved toward models that integrate multiple data types and scales. The multimodal foundation transformer models for multiscale genomics represent an emerging direction where models incorporate DNA sequence, RNA expression, chromatin state, and other modalities. Similarly, large-scale data-driven pre-trained DNA models continue to expand the scale of pretraining data to improve performance across diverse genomics tasks.

At a Glance: Foundation Models in Genomics

Practical Workflow for Using Genomic Foundation Models

Step 1: Define the Task and Required Output

Before selecting a model, specify the biological question and the type of prediction needed. Sequence classification, gene expression prediction, variant effect quantification, and regulatory region identification require different model capabilities. The benchmark study of DNA foundation models evaluated DNABERT-2, Nucleotide Transformer V2, HyenaDNA, Caduceus-Ph, and GROVER across sequence classification, gene expression prediction, variant effect quantification, and topologically associating domain recognition. Model performance varied among tasks and datasets, so no single model dominated across all applications.

Step 2: Select an Appropriate Model

Consider the training data composition relative to your target organism and task. Models trained on human genomes may transfer poorly to non-human species. The Nucleotide Transformer includes multi-species training data, which may improve cross-species generalization. For single-cell analysis, consider whether the model was pretrained on data from the relevant tissue or organism. The plant single-cell genomics review notes that foundation models address challenges specific to plant datasets including cell-type annotation and gene network modeling.

Step 3: Prepare Input Data

Format your data according to the model's expected input. DNA sequence models typically require tokenized sequences of specific lengths. Single-cell models require expression matrices in the format used during pretraining. The MutBERT approach demonstrates that how you represent the genome matters. Representing the genome as a probabilistic distribution over observed allele frequencies instead of raw sequence improved performance and efficiency.

Step 4: Fine-Tune or Use Zero-Shot

Decide whether to fine-tune the model on labeled data from your specific task or use zero-shot predictions. The Nucleotide Transformer study showed that models can be fine-tuned at low cost for various genomics applications. The ETHOS model for health trajectory prediction uses zero-shot learning, eliminating the need for labeled data and model fine-tuning. Zero-shot approaches save time and resources but may underperform fine-tuned models on tasks where labeled data is available.

Step 5: Evaluate Performance on Your Data

The benchmark study found that mean token embedding consistently improved sequence classification performance compared to other pooling strategies. Model performance varied among tasks and datasets. General-purpose DNA foundation models showed competitive performance in pathogenic variant identification but were less effective in predicting gene expression and identifying putative causal QTLs compared to specialized models. Evaluate multiple models on a held-out validation set from your own data before committing to one approach.

Options and Tradeoffs in Model Selection

Parameter Scale and Computational Cost

Larger models generally capture more complex patterns but require more computational resources for training and inference. The Nucleotide Transformer ranges from 50 million to 2.5 billion parameters. Smaller models may be sufficient for simpler tasks and are more practical in resource-constrained settings. The MutBERT study emphasizes that efficient representation strategies can achieve top performance without the largest model sizes.

Species and Data Domain Match

Training data composition directly affects model performance on your target data. Models trained predominantly on human genomes may not capture species-specific regulatory patterns. The Evo model trained on prokaryotic and phage genomes is well-suited for microbial genomics but may not transfer to human applications. The plant single-cell review highlights that foundation models are being adapted to address plant-specific challenges including crop resilience and stress adaptation.

Embedding and Pooling Strategies

How you extract representations from the model affects downstream performance. The benchmark study found that mean token embedding outperformed other pooling strategies for sequence classification. Different pooling strategies may be optimal for different tasks, so experiment with multiple approaches when building your pipeline.

Specialized Versus General-Purpose Models

The benchmark study found that general-purpose DNA foundation models were competitive for pathogenic variant identification but less effective for gene expression prediction and causal QTL identification compared to specialized models. If your task has a well-established specialized model, test it against general-purpose foundation models instead of assuming the foundation model will perform better.

Observations and Measurements in Model Evaluation

Benchmarking Across Tasks

The comprehensive benchmark provides a framework for model selection by evaluating five models across diverse genomic and genetic tasks. Key observations include the consistent improvement from mean token embedding for sequence classification and the task-dependent variation in model performance. When evaluating models for your own work, use multiple tasks relevant to your research question instead of relying on a single benchmark score.

Attribution and Interpretability

The SIGnature framework demonstrates that attribution scores from single-cell foundation models can reduce technical noise and emphasize regulatory genes. These attributions enable cross-dataset comparison, which is a core challenge in single-cell analyses. When using foundation models, consider whether attribution or interpretability tools are available for the model you select.

Population-Level Prediction

The Tissueformer model addresses a limitation of single-cell foundation models by predicting population-level labels from groups of single-cell RNA profiles while retaining single-cell resolution. Applied to COVID-19 severity prediction and cortical area identity prediction, Tissueformer outperformed single-cell foundation models and machine learning methods applied to pseudobulk and cell type composition. This approach enables automated construction of high-resolution brain region maps and quantification of individual differences in neuroanatomy.

Records and Documentation for Reproducibility

Track Model Versions and Training Data

Foundation models are released in versions with specific training data compositions. Document which model version you used, the exact training data described in the model card or publication, and any fine-tuning steps applied. The Nucleotide Transformer and MutBERT publications describe their training data and architectures in detail, which supports reproducibility.

Record Hyperparameters and Embedding Strategies

The benchmark study showed that embedding strategies significantly affect performance. Record the pooling method, sequence length, tokenization approach, and any fine-tuning hyperparameters in your analysis code and methods section.

Document Data Processing Steps

The MutBERT study demonstrates that representation strategy matters. Document how you processed raw sequencing data into model inputs, including any filtering, normalization, or probabilistic representation steps. This documentation is essential for others to reproduce your results.

Use Version Control and Containerization

Maintain your analysis code in version control and consider containerizing your environment to ensure that model versions and dependencies remain consistent. This practice supports reproducibility as models are updated or deprecated.

Common Failure Patterns and How to Avoid Them

Mismatch Between Training and Application Data

A frequent failure is applying a model trained on one species or data type to a different domain without validation. The Evo model trained on prokaryotic genomes may not perform well on human regulatory genomics. Always validate model performance on a representative sample of your target data before large-scale application.

Overlooking Embedding Strategy

The benchmark study found that pooling strategy significantly affects sequence classification performance. Researchers who use default settings without testing alternative embedding strategies may obtain suboptimal results. Test mean token embedding, CLS token embedding, and other pooling approaches on your validation set.

Assuming One Model Works for All Tasks

The benchmark study found that model performance varies among tasks and datasets. A model that performs well for variant effect prediction may perform poorly for gene expression prediction. Select models based on your specific task and validate empirically instead of relying on general reputation.

Ignoring Data Sparsity and Redundancy

The MutBERT study highlights that SNPs are rare in humans and standard masked language models may struggle to learn these variations efficiently. Training on entire datasets without prioritizing regions of genetic variation results in inefficiencies and negligible performance gains. Consider whether your data has similar sparsity issues and whether probabilistic representation approaches would help.

Neglecting Validation on Independent Data

Foundation models may memorize patterns from training data that do not generalize. The Nucleotide Transformer study emphasizes accurate predictions in low-data settings, but this claim requires validation on your own independent datasets. Always hold out data that was not used for fine-tuning or model selection.

Limitations and Interpretation Boundaries

Performance Variation Across Tasks

The benchmark study found that general-purpose DNA foundation models were less effective in predicting gene expression and identifying putative causal QTLs compared to specialized models. Foundation models are not universal solutions. For some tasks, specialized models trained on task-specific data remain superior.

Computational Resource Requirements

Large foundation models require substantial computational resources for fine-tuning and inference. The Nucleotide Transformer includes models up to 2.5 billion parameters. Researchers without access to high-performance computing may need to use smaller models or cloud resources, which affects model selection and experimental design.

Interpretability Challenges

Foundation models are complex and their predictions can be difficult to interpret. The SIGnature framework provides attribution scores that help interpret single-cell foundation model predictions, but similar tools are not available for all models. When interpretability is critical for your application, select models with available attribution or explanation tools.

Data Availability and Access

Training data for foundation models comes from public repositories. The EMBL-EBI Training and NCBI Data Resources provide access to genomic datasets used in model development. Researchers should verify data access conditions and licensing before using models or data in commercial or clinical applications.

Safety and Regulatory Context

Genomic Data Sharing Policies

Research using genomic foundation models often involves sensitive human genetic data. The NIH Genomic Data Sharing Policy governs how genomic data from NIH-funded research is shared and accessed. Researchers must comply with these policies when using human genomic data for model training, fine-tuning, or evaluation.

FAIR Data Principles

The FAIR Guiding Principles describe standards for making data findable, accessible, interoperable, and reusable. Applying these principles to your training data, model configurations, and analysis code supports reproducibility and enables others to build on your work. The principles are particularly relevant for foundation models where training data composition affects model behavior.

Clinical Translation Considerations

Foundation models are increasingly applied to health trajectory prediction and clinical decision support. The ETHOS model demonstrates zero-shot health trajectory prediction using transformer architectures. The AI-augmented precision prevention review discusses integrating mechanistic metabolic networks and multimodal biomarkers for early cardio-metabolic risk prediction. These applications raise questions about model validation, regulatory approval, and clinical deployment that researchers must consider when developing or applying models for health-related predictions.

Variant Classification Standards

The ACMG/AMP variant classification guidelines provide a framework for interpreting genetic variants. The study modeling these guidelines as a Bayesian classification framework found that most combining criteria were compatible but identified inconsistencies in some evidence combinations. When using foundation models for variant effect prediction, results should be interpreted within established classification frameworks instead of replacing them.

Professional Escalation Criteria

When to Consult Domain Experts

Foundation model predictions should be validated by domain experts before informing clinical or agricultural decisions. If a model predicts a variant effect, gene function, or health trajectory that conflicts with established biological knowledge, consult a specialist in the relevant domain before acting on the prediction.

When to Seek Computational Support

If you encounter computational resource limitations, unexpected model behavior, or difficulties reproducing published results, seek support from bioinformatics core facilities or computational biology collaborators. The EMBL-EBI Training and NCBI Data Resources offer educational materials and documentation that may help troubleshoot common issues.

When to Re-evaluate Model Selection

If model performance on your validation data falls below acceptable thresholds, re-evaluate your model selection. The benchmark study provides a framework for comparing models across tasks. Consider whether a specialized model might outperform a general-purpose foundation model for your specific application.

When to Report Concerns

If you identify systematic biases, unexpected failure modes, or potential safety concerns in foundation model predictions, report these findings through appropriate channels. This may include publishing validation studies, contacting model developers, or notifying institutional review boards for clinical applications.

Frequently Asked Questions

What is a foundation model in genomics?

A foundation model in genomics is a large-scale machine learning system pretrained on broad genomic data that can be adapted to multiple downstream tasks. These models learn general patterns from unlabeled DNA sequences, RNA expression profiles, or other biological data during pretraining. After pretraining, they can be fine-tuned on specific tasks with relatively small amounts of labeled data. The Nucleotide Transformer is an example of a foundation model pretrained on DNA sequences from thousands of human genomes and hundreds of diverse species.

How do foundation models differ from traditional bioinformatics approaches?

Traditional bioinformatics approaches typically build task-specific models from labeled training data for each new problem. Foundation models are pretrained on large amounts of unlabeled data and can be adapted to many tasks. This approach is particularly valuable in genomics where labeled data is often limited. The Nucleotide Transformer study showed that foundation models enable accurate predictions even in low-data settings and can be fine-tuned at low cost for various applications.

What tasks can genomic foundation models perform?

Genomic foundation models can perform sequence classification, gene expression prediction, variant effect quantification, regulatory region identification, cell-type annotation, and gene network modeling. The benchmark study evaluated five models across these diverse tasks. The Evo model demonstrated additional capabilities including zero-shot function prediction and generation of functional biological sequences.

How do I choose the right foundation model for my research?

Consider your task type, target species, available computational resources, and whether interpretability tools are needed. The benchmark study found that model performance varies among tasks and datasets, so evaluate multiple models on your own validation data. General-purpose models may be competitive for variant identification but less effective for gene expression prediction compared to specialized models.

What are the limitations of genomic foundation models?

Foundation models require substantial computational resources, their predictions can be difficult to interpret, and they may not generalize across species or data types. The benchmark study found that general-purpose models were less effective for gene expression prediction and causal QTL identification compared to specialized models. The MutBERT study highlighted that standard masked language models may struggle with sparse variation data.

How are foundation models used in single-cell genomics?

Single-cell foundation models are pretrained on large atlases of single-cell RNA-seq data and learn representations of cellular states. They are used for cell-type annotation, gene network modeling, and multi-omics integration. The plant single-cell genomics review describes how transformer-based architectures address challenges including data sparsity and dropout events. The SIGnature framework uses attribution scores from these models to score gene importance and enable cross-dataset comparison.

Can foundation models predict health trajectories?

Yes, foundation models are being applied to health trajectory prediction. The ETHOS model uses transformer architecture to analyze patient health timelines and predict future health trajectories using zero-shot learning. The Tissueformer model predicts population-level phenotypes from single-cell RNA profiles, including COVID-19 severity. The AI-augmented precision prevention review discusses integrating genomics, metabolomics, proteomics, and other data sources for cardio-metabolic risk prediction.

What data resources support foundation model research in genomics?

The EMBL-EBI Training provides educational materials for bioinformatics and genomics. The NCBI Data Resources provide access to genomic datasets used in model development and evaluation. The NIH Genomic Data Sharing Policy governs how genomic data from NIH-funded research is shared and accessed. The FAIR Guiding Principles describe standards for making data findable, accessible, interoperable, and reusable.

Related Bioinformatics Guides

• Single-Cell RNA-Seq Analysis Pipelines for Veterinary Immunology
• Single-cell RNA-seq Trajectory Inference and Cell Lineage Tracing
• Single-Cell RNA-Seq Normalization: Batch Effect Correction and Dimension Reduction (PCA, t-SNE, UMAP)
• Foundation Models for Single-Cell Biology
• Alternative Splicing Analysis from RNA-Seq Data

References and Further Reading

• EMBL-EBI Training. European Bioinformatics Institute.
• NCBI Data Resources. National Center for Biotechnology Information.
• Genomic Data Sharing Policy. National Institutes of Health.
• The FAIR Guiding Principles. Scientific Data.
• Advancing plant single-cell genomics with foundation models.. Current opinion in plant biology, 2024.
• Modeling the ACMG/AMP variant classification guidelines as a Bayesian classification framework.. Genetics in medicine : official journal of the American College of Medical Genetics, 2018.
• Sequence modeling and design from molecular to genome scale with Evo.. Science (New York, N.Y.), 2024.
• MutBERT: probabilistic genome representation improves genomics foundation models.. Bioinformatics (Oxford, England), 2025.
• CRISPR-based functional genomics tools in vertebrate models.. Experimental & molecular medicine, 2025.
• Nucleotide Transformer: building and evaluating robust foundation models for human genomics.. Nature methods, 2025.
• Spatially resolved clonal copy number alterations in benign and malignant tissue.. Nature, 2022.
• Benchmarking DNA foundation models for genomic and genetic tasks.. Nature communications, 2025.
• Scoring gene importance by interpreting single-cell foundation models.. 2026.
• Tissueformer: extending single-cell foundation models to predict population-level phenotypes.. 2026.
• AI-augmented precision prevention: Integrating mechanistic metabolic networks and&nbsp,multimodal biomarkers for early cardio-metabolic risk prediction. Journal of Preventive Medicine and Holistic Health, 2026.
• Zero shot health trajectory prediction using transformer. npj Digital Medicine, 2024.
• An Innovative Transformer-Based Approach for State of Health Trajectory Prediction and Remaining Useful Life Estimation in Lithium-Ion Batteries. International Conference on Clean Electrical Power, 2025.
• Deep Learning Regression with Sequences of Different Length: An Application for State of Health Trajectory Prediction and Remaining Useful Life Estimation in Lithium-Ion Batteries. Batteries, 2024.
• State of Health Trajectory Prediction Based on Multi-Output Gaussian Process Regression for Lithium-Ion Battery. Batteries, 2022.
• High-performance remaining useful life prediction for aeroengine based on combining health states and trajectory similarity. Engineering applications of artificial intelligence, 2025.
• Adapting Generative Genome Foundation Model Evo for Functional Genomics Prediction via Progressive Fine-Tuning. IEEE Transactions on Computational Biology and Bioinformatics, 2026.
• Multimodal foundation transformer models for multiscale genomics. Nature Methods, 2026.
• Large-scale data-driven pre-trained DNA models enhance performance across diverse genomics tasks. Nature Communications, 2026.
• Theories and models for genomics education and training. Genomic Medicine Skills and Competencies, 2022.

This article is educational and does not replace validated analysis plans, institutional policy, clinical interpretation, or specialist review.