Foundation Models in Genetics: Opportunities and Challenges

Foundation models are large-scale artificial intelligence systems pretrained on vast datasets through self-supervised learning, and in genetics they are being applied to predict gene expression, interpret genetic variants, annotate cell types, and generate functional DNA sequences. For geneticists, bioinformaticians, and life-science professionals, these models offer a path to extract biological meaning from the growing volume of genomic data, but their practical utility depends on careful task selection, rigorous benchmarking, and awareness of their limitations. This article examines what foundation models are, how they are built and evaluated, where they show genuine promise in genetics, and the challenges that must be managed before they can be trusted in research and clinical workflows.

What Is a Foundation Model in Genetics

A foundation model is a deep learning system pretrained on large, diverse datasets using self-supervised objectives, after which it can be adapted to many downstream tasks with minimal task-specific labeled data. In natural language processing, models such as Generative Pre-trained Transformers (GPT) and Bidirectional Encoder Representations from Transformers (BERT) established this paradigm by learning statistical patterns from massive text corpora. Researchers have extended the same approach to genomics, treating DNA sequences as a language and training transformer architectures to learn representations of nucleotides, genes, and regulatory elements.

The defining feature of a foundation model is its generality. Unlike traditional supervised models trained for a single task, foundation models are built once on broad data and then fine-tuned or probed for multiple applications. In single-cell genomics, for example, foundation models such as Geneformer and scGPT use transformer architectures to incorporate diverse omics data and extract latent patterns at both cell and gene levels, enabling analysis of cellular heterogeneity and complex regulatory networks. These models address challenges including improved cell-type annotation, gene network modeling, and multi-omics integration.

The scale of these models varies widely. The Nucleotide Transformer family ranges from 50 million up to 2.5 billion parameters and integrates information from 3,202 human genomes and 850 genomes from diverse species. Evo, a long-context genomic foundation model, was trained on millions of prokaryotic and phage genomes and can generalize across DNA, RNA, and proteins. These models learn context-specific representations of nucleotide sequences that allow for accurate predictions even in low-data settings.

The Data Landscape That Drives Genomic Foundation Models

Foundation models in genetics are only as capable as the data they are trained on, and the field benefits from a rich ecosystem of public genomic resources. The National Center for Biotechnology Information (NCBI) maintains extensive databases of DNA sequences, genes, and genetic variation that serve as primary training and validation data for many models. The European Bioinformatics Institute (EMBL-EBI) provides complementary training resources and bioinformatics education that help researchers understand how to work with these datasets effectively.

The scale of available data has grown rapidly due to high-throughput omics technologies, which have created exponential growth in biological data that often outpaces our ability to derive molecular insights. Population-scale projects such as the 1000 Genomes Project provide dense variation data across human populations, while large biobanks link genetic information to health outcomes. Single-cell genomics has generated repositories of cellular-level measurements that require unified frameworks for integration and analysis.

However, the characteristics of genomic data create specific challenges for foundation model training. Human population data are sparse and redundant, with single nucleotide polymorphisms being rare events across the genome. Current masked language models trained directly on whole-genome sequences may struggle to efficiently learn these variations, and training on the entire dataset without prioritizing regions of genetic variation results in inefficiencies and negligible gains in performance. The MutBERT model addresses this by representing the entire genome as a probabilistic distribution over observed allele frequencies, focusing on informative genomic variations while maintaining computational efficiency.

Core Principles of Genomic Foundation Model Design

Self-Supervised Pretraining on DNA Sequences

The pretraining objective determines what a foundation model learns from raw sequence data. Most genomic foundation models use masked language modeling, where portions of the DNA sequence are hidden and the model learns to predict the masked nucleotides from surrounding context. This forces the model to capture dependencies between sequence positions, including coding structure, regulatory motifs, and evolutionary conservation.

The choice of tokenization is a critical design decision. DNA can be tokenized at the nucleotide level, as k-mer chunks, or using learned vocabularies that reflect functional units. The Nucleotide Transformer models produce context-specific representations of nucleotide sequences, which allow for accurate predictions even in low-data settings. Despite receiving no supervision about genomic function, these models learned to focus attention on key genomic elements such as promoters, enhancers, and coding regions.

Scaling Laws and Model Architecture

Research on genomic foundation models has identified scaling laws that complement observations in language and vision. Evo reported scaling laws on DNA, showing that performance improves predictably with model size and training data. However, the relationship between scale and utility is not uniform across tasks. Evaluation of single-cell foundation models Geneformer and scGPT found that the benefits of large-scale pretraining are strongly task-dependent, conferring substantial advantages in cell type annotation but limited gains in perturbation prediction. Increasing model size does not guarantee improved performance and can even be detrimental, challenging the assumption that bigger models are always better.

Probabilistic Representations for Variation

Standard masked language models treat each position as a discrete token, which does not reflect the probabilistic nature of population variation. MutBERT introduced a probabilistic genome representation that encodes observed allele frequencies as a distribution over possible nucleotides. This approach efficiently utilizes SNP information from population-scale genomic data and consistently ranked among the top-performing models across multiple downstream prediction tasks when compared against DNABERT-2, various versions of Nucleotide Transformer, and modified versions of MutBERT.

At a Glance: Representative Genomic Foundation Models

Applications of Foundation Models in Genetics

Predicting Gene Expression from DNA Sequence

One of the most active applications of genomic foundation models is predicting molecular phenotypes from DNA sequences. This task has been a longstanding challenge in genomics, often driven by limited annotated data and the inability to transfer learnings between tasks. Foundation models address this by pretraining on unlabeled sequence data, then fine-tuning on smaller labeled datasets for specific prediction tasks.

The Nucleotide Transformer study demonstrated that these models can be fine-tuned at low cost to solve a variety of genomics applications, including gene expression prediction. However, benchmarking studies reveal important caveats. A comprehensive evaluation of five DNA foundation models (DNABERT-2, Nucleotide Transformer V2, HyenaDNA, Caduceus-Ph, and GROVER) found that general purpose DNA foundation models showed competitive performance in pathogenic variant identification but were less effective in predicting gene expression and identifying putative causal QTLs compared to specialized models.

Variant Effect Prediction and Pathogenicity Assessment

Predicting the functional impact of genetic variants is central to clinical genetics and genomic medicine. Foundation models offer a new approach to variant prioritization by learning sequence context that influences variant effects. The Nucleotide Transformer models can be used to improve the prioritization of genetic variants without explicit supervision about pathogenicity.

The clinical variant interpretation process itself has been formalized through guidelines from the American College of Medical Genetics and Genomics and the Association for Molecular Pathology (ACMG/AMP). These guidelines were evaluated for internal consistency and compatibility with Bayesian statistical reasoning, and the criteria were translated into a naive Bayesian classifier with exponentially scaled odds of pathogenicity. This quantitative framework validates the approach adopted by the ACMG/AMP and supports efforts to automate components of variant pathogenicity assessments. Foundation models can contribute evidence to this framework, but they do not replace the structured reasoning process that the guidelines encode.

Single-Cell Genomics and Cell Type Annotation

Single-cell genomics has produced massive datasets that require scalable analysis methods. Single-cell foundation models (scFMs) use transformer architectures to incorporate diverse omics data and extract latent patterns at both cell and gene levels. These models address challenges in plant single-cell genomics including improved cell-type annotation, gene network modeling, and multi-omics integration.

Evaluation of scFMs across multiple tasks shows that their utility varies substantially by application. In cell type annotation, large-scale pretraining confers substantial advantages. In perturbation prediction, however, the tested models may capture little more than simple summary statistics, suggesting limited capacity to learn complex biological interactions. This finding highlights the need to move beyond scaling and toward developing models that integrate deeper biological knowledge.

Generating Functional Sequences and Designing Biological Systems

Generative foundation models can produce new DNA, RNA, and protein sequences with desired properties. Evo demonstrated the generation of functional CRISPR-Cas and transposon systems, representing early examples of protein-RNA and protein-DNA codesign with a language model. The model also generates megabase-scale sequences with plausible genomic architecture and learns how small mutations affect whole-organism fitness.

In the medical domain, generative AI can produce realistic protein sequences, genetic variants, or clinical notes. Large language models trained on biomedical literature and electronic health records can summarize clinical findings, identify patterns, and provide decision support for clinicians. Synthetic data generation can help protect patient privacy and augment existing disease datasets.

Multimodal Integration Across Omics Layers

The future direction of foundation models in genetics points toward multimodal integration. Researchers envision developing multimodal foundation models pretrained on diverse omics datasets, including genomics, transcriptomics, epigenomics, proteomics, metabolomics, and spatial profiling. These models are expected to characterize the molecular states of cells across a broad continuum, facilitating the creation of holistic maps of cells, genes, and tissues.

Context-specific transfer learning of foundation models can empower diverse applications from novel cell-type recognition, biomarker discovery, and gene regulation inference to in silico perturbations. Multimodal foundation transformer models for multiscale genomics represent an emerging approach to integrating information across genomic scales, from individual variants to chromosome organization.

Practical Workflow for Using Foundation Models in Genetics Research

Step 1: Define the Biological Question and Task Type

Before selecting a foundation model, clarify whether the task is sequence classification, gene expression prediction, variant effect quantification, cell type annotation, or sequence generation. Benchmarking studies show that model performance varies among tasks and datasets, so the optimal model for one application may be suboptimal for another. General purpose DNA foundation models may excel at pathogenic variant identification while specialized models perform better for gene expression prediction.

Step 2: Select an Appropriate Model and Embedding Strategy

The choice of model architecture, pretraining data, and embedding strategy significantly impacts performance. Mean token embedding consistently and significantly improves sequence classification performance compared to other pooling strategies. Consider whether the model was trained on data relevant to your organism and genomic context. Models trained on human genomes may not transfer well to plant or microbial applications, and vice versa.

Step 3: Evaluate with Zero-Shot Embeddings Before Fine-Tuning

Many foundation models can be evaluated using zero-shot embeddings, where the pretrained model produces representations without task-specific training. This approach allows rapid assessment of whether the model has learned relevant features for your task. If zero-shot performance is inadequate, fine-tuning on task-specific labeled data may improve results, but this requires sufficient annotated examples and computational resources.

Step 4: Benchmark Against Specialized Models and Baselines

Foundation models should not be assumed superior to specialized models without direct comparison. The comprehensive benchmark of DNA foundation models provides a framework for model selection, highlighting the impact of architecture, pretraining data, and embedding strategies on performance. Compare foundation model outputs against established tools for your specific task, and report both performance metrics and computational costs.

Step 5: Validate Predictions with Independent Experiments

Computational predictions from foundation models require experimental validation before they inform biological conclusions. In the CAR-T cell therapy example, an LLM-guided in silico platform identified IL-15 as the most effective cytokine enhancer for liver cancer CAR-T cells, and this prediction was validated through in vitro and in vivo experiments. The recovery of IL-15 served as a positive benchmark supporting the validity of the workflow, but the validation step was essential.

Records and Measurements for Foundation Model Workflows

Maintaining detailed records of foundation model usage is essential for reproducibility and for building confidence in results. Document the model version, pretraining data description, fine-tuning parameters, embedding strategy, and evaluation metrics for every analysis. Record the computational resources required, including GPU hours and memory usage, since these costs influence the feasibility of different approaches.

For variant effect prediction, record the specific genomic coordinates, reference and alternate alleles, and the model's prediction score. Compare these scores against ACMG/AMP evidence categories where applicable, noting that the Bayesian framework underlying these guidelines provides a mathematical foundation for combining evidence types. Track how often foundation model predictions agree with or contradict established variant interpretation methods.

For single-cell analyses, document the preprocessing pipeline, the specific foundation model used, and the downstream task configuration. Note whether the model was used in zero-shot mode or fine-tuned, and record the training data composition if fine-tuning was performed. The evaluation of scFMs on real versus synthetic data with different levels of complexity can reveal whether models are learning meaningful biology or simple summary statistics.

Common Failure Patterns and How to Avoid Them

Overestimating Model Generalization Across Tasks

A frequent error is assuming that a foundation model that performs well on one task will perform well on all tasks. Benchmarking studies consistently show that model performance varies among tasks and datasets. The same model that excels at cell type annotation may fail at perturbation prediction. Evaluate models on your specific task instead of relying on published performance from different applications.

Ignoring Data Distribution Mismatches

Foundation models trained on one data distribution may underperform when real-world conditions change. This is a documented risk for generative AI in medicine, where models can produce convincing but incorrect results when applied to populations or conditions outside their training data. Check whether your data resemble the model's pretraining data in terms of species, sequencing technology, genomic region, and population ancestry.

Treating Model Outputs as Ground Truth

Foundation model predictions are statistical inferences, not experimental measurements. The ACMG/AMP guidelines provide a structured framework for variant classification precisely because computational predictions alone are insufficient for clinical decisions. Use foundation model outputs as one line of evidence among many, and validate critical predictions experimentally.

Neglecting Computational Costs

Training and fine-tuning foundation models require substantial computational resources. The Nucleotide Transformer models range up to 2.5 billion parameters, and training such models demands specialized hardware. Even inference with large models can be computationally intensive. Assess whether the performance gains justify the computational costs compared to lighter-weight specialized models.

Assuming Larger Models Are Always Better

The evaluation of Geneformer and scGPT found that increasing model size does not guarantee improved performance and can even be detrimental for some tasks. This challenges the assumption that scaling alone will solve genomics problems. Consider task-specific architectures and biologically informed priors as alternatives to simply increasing model size.

Limitations and Interpretation Boundaries

Biological Complexity Beyond Sequence Context

Foundation models learn statistical patterns from sequence data, but many biological processes depend on context that is not captured in sequence alone. Gene expression is influenced by chromatin state, three-dimensional genome organization, cellular environment, and developmental timing. Spatially resolved transcriptomics has revealed clonal copy number alterations in benign and malignant tissues, demonstrating that genomic alterations must be understood in spatial and temporal context. Foundation models trained only on sequence data cannot capture these dimensions.

The Nonsequential Nature of Omics Data

Unlike natural language, which has a linear structure, omics data are inherently nonsequential. Genes interact in networks, regulatory elements act at variable distances, and cellular states are defined by multidimensional molecular profiles. Single-cell foundation models face challenges including the nonsequential nature of omics data, inconsistency in data quality, and the computational intensity required for training and fine-tuning.

Interpretability Gaps

Interpreting the biological relevance of latent embeddings and model representations remains nontrivial. Foundation models learn internal representations that correlate with biological features, but the mapping between model internals and biological mechanisms is often opaque. This limits the ability to extract mechanistic insights from model predictions and complicates the communication of results to clinical or regulatory audiences.

Hidden Biases in Training Data

Generative AI models carry major risks including reflecting hidden biases in the training data. Genomic datasets are not representative of all human populations, with some ancestries overrepresented and others nearly absent. Models trained on biased data may produce systematically incorrect predictions for underrepresented groups. This is particularly concerning for clinical applications where variant interpretation affects medical decisions.

Safety and Regulatory Context

Clinical Genetics Workforce Readiness

The integration of AI into clinical genetics is proceeding faster than workforce training. A survey of 215 US-based genetics clinicians and trainees found that over half report little to no knowledge of AI in clinical genetics, and 64.3% reported no formal training. Most participants reported insufficient knowledge of clinical AI, desired more education, and would take available training. This gap between technological capability and workforce readiness is a critical constraint on clinical adoption.

Data Sharing and Privacy

Genomic data are sensitive personal information, and their use in foundation model training raises privacy concerns. The National Institutes of Health Genomic Data Sharing Policy establishes expectations for responsible sharing of genomic data generated through NIH-funded research. Synthetic data generation using generative AI can help protect patient privacy and augment existing disease datasets, but the privacy guarantees of synthetic data require careful evaluation.

FAIR Data Principles

The FAIR Guiding Principles provide a framework for making data findable, accessible, interoperable, and reusable. These principles are directly relevant to foundation model development, which depends on large-scale data sharing and integration. Adherence to FAIR principles enables the assembly of diverse training datasets and supports the benchmarking and validation that foundation models require.

Validation and Regulation of AI in Medicine

Generalist medical AI models will challenge current strategies for regulating and validating AI devices for medicine. Foundation models that can perform diverse tasks with little or no task-specific labeled data do not fit traditional validation frameworks that assume fixed inputs and outputs. The development of appropriate validation standards is an active area of discussion, and researchers should be aware that regulatory expectations for AI in clinical genetics are still evolving.

Professional Escalation Criteria

Researchers and clinicians using foundation models in genetics should establish clear criteria for when to escalate concerns to supervisors, institutional review boards, or regulatory authorities. Escalate when foundation model predictions are used for clinical decisions without experimental validation, when models produce results that contradict established biological knowledge, or when model outputs could affect patient care. Escalate when training data include sensitive genomic information without appropriate consent or data sharing approvals. Escalate when model performance degrades significantly across population subgroups, suggesting bias in training data or model architecture.

For clinical applications, escalate any variant interpretation that relies on foundation model predictions without confirmation through orthogonal methods. The ACMG/AMP framework provides a structured approach to variant classification, and deviations from this framework require documented justification. Escalate when foundation model outputs are presented as definitive without acknowledgment of their probabilistic nature.

Frequently Asked Questions

What exactly is a foundation model in genetics?

A foundation model in genetics is a large-scale deep learning model pretrained on vast genomic datasets through self-supervised learning, after which it can be adapted to various downstream tasks. These models use transformer architectures to learn representations of DNA, RNA, or protein sequences that capture biological patterns. Examples include Nucleotide Transformer for human genomics, Evo for prokaryotic genomes, and Geneformer and scGPT for single-cell data.

How do foundation models differ from traditional bioinformatics tools?

Traditional bioinformatics tools are typically designed for a single task, such as variant calling, sequence alignment, or gene annotation. Foundation models are pretrained on broad data and can be applied to many tasks with minimal task-specific labeled data. They learn general representations of genomic sequences that can be fine-tuned for specific applications, whereas traditional tools encode domain knowledge directly in their algorithms.

Can foundation models predict the clinical significance of genetic variants?

Foundation models can prioritize genetic variants by learning sequence context that influences variant effects, and the Nucleotide Transformer models have been shown to improve variant prioritization. However, clinical variant interpretation requires structured evidence evaluation according to ACMG/AMP guidelines, which have been formalized as a Bayesian classification framework. Foundation model predictions should be considered one line of evidence within this framework, not a replacement for it.

What are the main limitations of current genomic foundation models?

Current limitations include task-dependent performance where models excel at some applications but fail at others, difficulty capturing biological context beyond sequence data, interpretability gaps in understanding what models have learned, hidden biases in training data, and substantial computational costs. Evaluation of single-cell foundation models found that increasing model size does not guarantee improved performance and can even be detrimental for some tasks.

How should I choose a foundation model for my genomics research?

Define your biological question and task type first, then select a model trained on data relevant to your organism and genomic context. Evaluate models using zero-shot embeddings before fine-tuning, and benchmark against specialized models and baselines. Consider the embedding strategy, as mean token embedding consistently improves sequence classification performance. Document model version, training data, and evaluation metrics for reproducibility.

Are foundation models ready for clinical use in genetics?

Clinical use of foundation models is still emerging, and the clinical genetics workforce reports limited knowledge and training in AI applications. A survey found that most clinicians desired more education and would take available training. While some AI applications such as facial diagnostic tools and AI-generated genomic testing results are already used in practice, foundation models require careful validation and regulatory consideration before they can be trusted for clinical decisions.

How do foundation models handle rare genetic variants?

Foundation models face challenges with rare variants because human population data are sparse, and masked language models trained directly on whole-genome sequences may struggle to efficiently learn these variations. Probabilistic representations such as MutBERT address this by encoding allele frequencies as distributions, focusing on informative genomic variations while maintaining computational efficiency. However, rare variant interpretation remains challenging and requires multiple lines of evidence.

What ethical considerations apply to foundation models in genetics?

Key ethical considerations include privacy of genomic data used in training, representativeness of training data across human populations, transparency about model limitations, and appropriate use in clinical decisions. The NIH Genomic Data Sharing Policy establishes expectations for responsible data sharing, and the FAIR Guiding Principles provide a framework for data findability, accessibility, interoperability, and reusability. Synthetic data generation may help protect privacy but requires careful evaluation of its guarantees.

Related Bioinformatics Guides

• Biological Foundation Models for Predicting Host Tropism of Zoonotic Viruses
• Ethical Considerations in Computational Genomics
• Predicting AMR from Genomic Data
• Foundation Models for Single-Cell Biology
• Variant Effect Prediction with Deep Mutational Scanning Data

References and Further Reading

• EMBL-EBI Training. European Bioinformatics Institute.
• NCBI Data Resources. National Center for Biotechnology Information.
• Genomic Data Sharing Policy. National Institutes of Health.
• The FAIR Guiding Principles. Scientific Data.
• Advancing plant single-cell genomics with foundation models.. Current opinion in plant biology, 2024.
• Modeling the ACMG/AMP variant classification guidelines as a Bayesian classification framework.. Genetics in medicine : official journal of the American College of Medical Genetics, 2018.
• Sequence modeling and design from molecular to genome scale with Evo.. Science (New York, N.Y.), 2024.
• MutBERT: probabilistic genome representation improves genomics foundation models.. Bioinformatics (Oxford, England), 2025.
• CRISPR-based functional genomics tools in vertebrate models.. Experimental & molecular medicine, 2025.
• Nucleotide Transformer: building and evaluating robust foundation models for human genomics.. Nature methods, 2025.
• Spatially resolved clonal copy number alterations in benign and malignant tissue.. Nature, 2022.
• Benchmarking DNA foundation models for genomic and genetic tasks.. Nature communications, 2025.
• Generative AI and Language Models in Human Genetics and Health: From Variant Interpretation to Clinical Decision Support.. 2026.
• LLMs in Medical Education for Autism Caregivers: A Comparative Evaluation of Accuracy, Readability, Actionability, and Neurodiversity-Affirming Language.. 2026.
• Initial-Visit Specialty Triage in Rare Diseases Using Large Language Models: Retrospective Benchmarking Study.. 2026.
• Artificial intelligence in clinical genetics: current practice and attitudes among the clinical genetics workforce.. 2026.
• Large language model-guided CAR-T in silico platform for cytokine optimization in liver cancer with low antigen density.. 2026.
• Evaluating the learnability of single-cell large language models on multiple tasks.. 2026.
• Foundation models for generalist medical artificial intelligence. Nature, 2023.
• Towards multimodal foundation models in molecular cell biology. Nature, 2025.
• Single-cell foundation models: bringing artificial intelligence into cell biology. Experimental and Molecular Medicine, 2025.
• Adapting Generative Genome Foundation Model Evo for Functional Genomics Prediction via Progressive Fine-Tuning. IEEE Transactions on Computational Biology and Bioinformatics, 2026.
• Multimodal foundation transformer models for multiscale genomics. Nature Methods, 2026.
• Large-scale data-driven pre-trained DNA models enhance performance across diverse genomics tasks. Nature Communications, 2026.
• Theories and models for genomics education and training. Genomic Medicine Skills and Competencies, 2022.

This article is educational and does not replace validated analysis plans, institutional policy, clinical interpretation, or specialist review.