Deep Learning-Driven Protein Language Models for Predicting Antigenic Drift in Influenza A Hemagglutinin

Introduction

Influenza A virus (IAV) remains a major pathogen in veterinary medicine, causing respiratory disease in poultry, swine, equids, and other mammalian hosts. The hemagglutinin (HA) glycoprotein is the primary target of host neutralizing antibodies and undergoes continuous antigenic drift through the accumulation of point mutations in epitope regions [1]. Traditional surveillance methods rely on phylogenetic analysis of HA sequences combined with serological assays such as hemagglutination inhibition (HI) tests to identify antigenically distinct variants [2]. However, these approaches are labor intensive and may lag behind the rapid emergence of drift variants in field populations.

Deep learning-driven protein language models (PLMs) offer a transformative approach to predicting antigenic drift directly from HA amino acid sequences. These models, including ESM-1b and ProtBERT, are trained on large corpora of protein sequences and learn distributed representations (embeddings) that capture evolutionary, structural, and functional constraints [3]. By applying these embeddings to HA sequences, researchers can quantify antigenic distances, identify emerging drift variants, and prioritize strains for vaccine updates in veterinary species. This article provides a detailed technical review of PLM-based methods for predicting antigenic drift in IAV HA, with emphasis on applications in veterinary virology and surveillance.

Protein Language Models and Embeddings

Protein language models are neural network architectures, typically based on the transformer encoder, that are pretrained on millions of protein sequences using a masked language modeling objective [3]. During pretraining, the model learns to predict masked amino acids in a sequence based on the surrounding context. This process forces the model to internalize patterns of coevolution, structural propensities, and functional constraints. The resulting per-residue embeddings are high-dimensional vectors (e.g., 1280 dimensions for ESM-1b) that encode both local and global sequence information.

For HA sequences, PLM embeddings capture information relevant to antigenic drift. Mutations in epitope regions alter the local sequence context, which is reflected in changes to the embedding vectors. By comparing embeddings from different HA sequences, one can compute a distance metric that correlates with antigenic dissimilarity. This approach bypasses the need for explicit multiple sequence alignments or structural modeling, though structural information can be integrated for improved accuracy.

The key advantage of PLMs over traditional sequence-based methods (e.g., pairwise identity, phylogenetic distance) is their ability to capture epistatic interactions and long-range dependencies. Antigenic drift often involves combinations of mutations that are individually neutral but collectively alter antibody binding [4]. PLM embeddings can represent such combinatorial effects because the model attends to all positions in the sequence simultaneously.

Training Data and Model Architecture

The training data for PLMs typically consist of sequences from the UniRef or Pfam databases, which include millions of protein families [3]. For veterinary applications, it is important that the training data include sufficient representation of viral glycoproteins, including IAV HA from diverse subtypes (H1-H18) and host species. However, the general protein language model is not fine-tuned on HA sequences alone; instead, the pretrained embeddings are used as features for downstream tasks.

The architecture of ESM-1b is a 33-layer transformer with approximately 650 million parameters [3]. ProtBERT is a smaller model with 12 layers and 420 million parameters, trained on the BFD database. Both models use self-attention mechanisms that allow each residue to attend to all other residues in the sequence. The attention weights can be extracted to identify which positions the model considers important for predicting masked residues. These attention maps have been shown to correlate with structural contacts and functional sites [5].

For antigenic drift prediction, the typical workflow involves:

1. Collecting HA sequences from public databases (e.g., GISAID, GenBank) with associated metadata including host species, subtype, and year of isolation.
2. Extracting per-residue embeddings from the penultimate layer of the PLM.
3. Computing a sequence-level embedding by averaging or pooling per-residue embeddings.
4. Training a classifier or regression model (e.g., support vector machine, random forest, or neural network) on a labeled dataset of antigenic distances derived from HI assays.
5. Validating the model on held-out data, including known drift events.

The following Mermaid diagram illustrates the workflow:

flowchart TD
    A[HA sequences from GISAID/GenBank], > B[Extract PLM embeddings (ESM-1b/ProtBERT)]
    B, > C[Compute sequence-level embedding (average pooling)]
    C, > D[Train classifier on HI-derived antigenic distances]
    D, > E[Validate on known drift events]
    E, > F[Predict antigenic distance for new sequences]
    F, > G[Identify emerging drift variants]
    G, > H[Update vaccine strains for veterinary use]

Attention Maps and Epitope Prediction

One of the most powerful features of transformer-based PLMs is the ability to visualize attention maps. Attention weights indicate how much each residue contributes to the representation of another residue. In the context of HA, attention maps can highlight residues that are coevolutionarily coupled, which often correspond to structurally proximal positions in the folded protein [5]. These maps can be used to predict epitope regions without requiring a crystal structure.

For example, attention heads in the deeper layers of ESM-1b have been shown to focus on residues in the receptor-binding site and antigenic sites (Sa, Sb, Ca1, Ca2, Cb) of HA [6]. By analyzing changes in attention patterns between sequences, researchers can identify mutations that disrupt or create new interactions, potentially leading to antigenic drift. This approach is complementary to structure-based methods such as those described in Machine Learning-Driven Prediction of Antigenic Drift in Influenza A Hemagglutinin Using Structural Dynamics and Sequence Surveillance.

Attention maps can also be used to prioritize residues for experimental validation via deep mutational scanning, as discussed in Deep Mutational Scanning and Machine Learning for Predicting Antibody Escape Mutations in Influenza A Virus. By combining PLM attention with structural modeling, one can generate hypotheses about which mutations are most likely to cause antigenic escape.

Validation Against Known Drift Events

Validation of PLM-based drift prediction requires a benchmark dataset of HA sequences with known antigenic phenotypes. In veterinary virology, such datasets are available for swine influenza A virus (IAV-S) and equine influenza virus (EIV). For example, the antigenic evolution of H3N8 equine influenza has been well characterized using HI assays [7]. Similarly, H1N1, H1N2, and H3N2 swine influenza viruses have been monitored antigenically in North America and Europe [8].

A typical validation study would:

• Collect HA sequences from a defined time period (e.g., 10 years) and corresponding HI titers against reference antisera.
• Compute antigenic distances using the PLM embedding method and compare them to HI-derived distances.
• Assess the ability of the PLM model to correctly classify sequences into antigenic clusters (e.g., using the metric of cluster purity).
• Evaluate the model's sensitivity to detect drift events that were identified retrospectively.

Results from such studies indicate that PLM embeddings achieve high correlation with HI distances, often outperforming sequence identity and phylogenetic distance [9]. The embeddings are particularly effective at capturing the antigenic impact of mutations in the globular head domain of HA, where most epitopes reside.

Comparison with Traditional Phylogenetic Methods

Traditional phylogenetic methods for antigenic drift prediction rely on constructing a tree from HA sequences and inferring antigenic clusters based on branch lengths or tree topology. However, phylogenetic distance does not always correlate with antigenic distance because synonymous mutations and neutral substitutions can inflate branch lengths without affecting antigenicity [10]. Moreover, phylogenetic methods require accurate multiple sequence alignments and are sensitive to recombination and selection pressures.

PLM embeddings offer several advantages:

• They are alignment-free, avoiding issues with gap placement and alignment uncertainty.
• They capture functional constraints directly from sequence context, not just evolutionary divergence.
• They can be updated incrementally as new sequences become available without rebuilding the entire tree.
• They provide per-residue resolution, enabling identification of specific mutations driving drift.

A comparison of methods is summarized in Table 1.

Table 1. Comparison of methods for predicting antigenic drift in HA.

PLM methods can be combined with structural approaches, as described in Structural Prediction and Evolutionary Dynamics of Avian Influenza Hemagglutinin Using Deep Learning and Molecular Dynamics. For example, PLM embeddings can be used to prioritize mutations for molecular dynamics simulations of antibody-HA binding.

Integration with Structural Surveillance

To maximize utility in veterinary surveillance, PLM predictions should be linked to structural visualization tools. By mapping attention weights or embedding distances onto a 3D structure of the HA trimer (e.g., from the Protein Data Bank), researchers can visualize predicted epitope changes. This integration allows veterinarians and virologists to quickly assess whether emerging mutations are located in antigenic sites.

A suggested workflow involves:

1. Retrieving the HA sequence from a field isolate.
2. Extracting PLM embeddings and computing attention maps.
3. Mapping high-attention residues onto a homologous HA structure using structural alignment.
4. Highlighting residues that differ from the current vaccine strain.
5. Generating a 3D visualization with color-coded attention scores.

This approach is analogous to the methods described in Computational Visualization of Single-Point Mutations on Protein 3D Structures. By combining PLM predictions with structural context, the biological relevance of predicted drift events becomes immediately apparent.

Conclusion

Deep learning-driven protein language models represent a powerful new tool for predicting antigenic drift in influenza A hemagglutinin. By learning rich sequence representations from large protein databases, these models capture evolutionary and structural constraints that are relevant to antigenic variation. PLM embeddings can be used to compute antigenic distances, identify emerging drift variants, and prioritize mutations for experimental characterization. Compared to traditional phylogenetic methods, PLMs offer alignment-free, position-specific predictions that can be updated rapidly as new sequences become available.

For veterinary virology, the application of PLMs to HA sequences from avian, swine, and equine influenza viruses holds promise for improving vaccine strain selection and surveillance. Integration with structural visualization tools further enhances interpretability. As PLM architectures continue to evolve, their accuracy and utility in predicting antigenic drift will likely increase, supporting proactive responses to emerging variants in animal populations.

References

[1] Swayne, D. E., & Suarez, D. L. (Eds.). (2020). Diseases of Poultry (14th ed.). Wiley-Blackwell.

[2] World Organisation for Animal Health (OIE). (2021). Manual of Diagnostic Tests and Vaccines for Terrestrial Animals. Chapter 3.3.4: Avian Influenza.

[3] Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., ... & Fergus, R. (2021). Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. Proceedings of the National Academy of Sciences, 118(15), e2016239118.

[4] Smith, D. J., Lapedes, A. S., de Jong, J. C., Bestebroer, T. M., Rimmelzwaan, G. F., Osterhaus, A. D., & Fouchier, R. A. (2004). Mapping the antigenic and genetic evolution of influenza virus. Science, 305(5682), 371-376.

[5] Vig, J., Madani, A., Varshney, L. R., Xiong, C., Socher, R., & Rajani, N. F. (2020). BERTology meets biology: Interpreting attention in protein language models. arXiv preprint arXiv:2006.15222.

[6] Hie, B., Zhong, E. D., Berger, B., & Bryson, B. (2021). Learning the language of viral evolution and escape. Science, 371(6526), 284-288.

[7] Daly, J. M., Newton, J. R., & Mumford, J. A. (2004). Current perspectives on control of equine influenza. Veterinary Research, 35(4), 411-423.

[8] Vincent, A. L., Ma, W., Lager, K. M., Janke, B. H., & Richt, J. A. (2008). Swine influenza viruses: a North American perspective. Advances in Virus Research, 72, 127-154.

[9] Meier, J., Rao, R., Verkuil, R., Liu, J., Sercu, T., & Rives, A. (2021). Language models enable zero-shot prediction of the effects of mutations on protein function. bioRxiv, 2021.07.09.450648.

[10] Bedford, T., Suchard, M. A., Lemey, P., Dudas, G., Gregory, V., Hay, A. J., ... & Rambaut, A. (2014). Integrating influenza antigenic dynamics with molecular evolution. eLife, 3, e01914. *** Disclaimer: This article is for educational and informational purposes only. It is not intended to substitute for professional veterinary advice, diagnosis, treatment, or regulatory guidance. Always consult a licensed veterinarian or qualified specialist regarding animal health, disease diagnosis, and therapeutic decisions.