Protein language model embeddings tested against classical descriptors for spotting resistance genes

The work

Technical · from the record

The study compared two alignment-free ways of representing bacterial proteins for antimicrobial resistance detection: hand-computed compositional and physicochemical descriptors, and frozen embeddings from the ESM-2 protein language model. Logistic regression, an SVM, a random forest, a feed-forward neural network and a stacked fusion of the four were fitted on a balanced, redundancy-filtered dataset and evaluated on a held-out test set of 908 proteins and an external cohort of 527 sequences. On the held-out test set, AUROC ran from 0.817 to 0.850 for the classical descriptors, while the fusion model on deep embeddings reached 0.966; on the external cohort the neural network on deep embeddings reached an AUROC of 0.9717, and concatenating the two feature sets scored lower than embeddings alone across matching models. Label-permutation and sequence-scrambling controls dropped performance to AUROC 0.492 and to a range of 0.57 to 0.65 respectively, and AlphaFold3 structures plus phylogenetics were used to examine 17 proteins misclassified across all three feature spaces.

How AI was used

AMR-positive proteins were pooled from CARD, AMRFinderPlus and ResFinder, length- and residue-filtered, clustered with CD-HIT at 95% identity, and balanced against RefSeq proteins screened for latent resistance homology, then split 70/15/15. Each sequence was encoded twice: as classical amino acid, dipeptide, tripeptide and physicochemical descriptors, and as a mean-pooled protein-level vector from a pretrained ESM-2 transformer used with fixed weights and no fine-tuning; the two were also concatenated. For each of the three feature spaces, logistic regression, an RBF support vector machine, a random forest, a feed-forward neural network and a stacking fusion model were fitted on the training partition inside scikit-learn pipelines, with hyperparameters and decision thresholds tuned on the validation partition, 10-fold cross-validation on the pooled training-validation space, repetition across 10 random seeds, and a 50-iteration randomized re-partitioning procedure. The locked models were then run without retraining on the internal test partition, on label-shuffled and window-scrambled control matrices built through the same feature pipeline, and on an external beta-lactamase and RefSeq cohort. Existing tools (DeepARG, PLM-ARG, ProtAlign-ARG, AMRFinderPlus) were run on the same test set for comparison, and AlphaFold3 was used to model structures of the proteins misclassified across all feature spaces.