Five structure-prediction models tested on 509 protein–peptide pairs under altered inputs

The work

Technical · from the record

The authors assembled a non-redundant set of 509 experimentally resolved protein-peptide complexes and predicted each one with AlphaFold2, AlphaFold2-Multimer, AlphaFold3, Boltz-1 and Chai-1, scoring the predictions against the experimental structures with DockQ. Comparing a 68-structure post-training-cutoff set with the 68 latest pre-cutoff structures, and separately counting how often each binding site appeared in each model's training data, they found lower accuracy for AlphaFold3, Boltz-1 and Chai-1 on post-cutoff structures and for binding sites with no match in the training set. A series of alignment manipulations showed little accuracy difference between paired and species-shuffled paired alignments, while removing the peptide alignment improved 17%-26% of predictions when present; of complexes predicted successfully with the peptide sequence supplied, 40%-51% remained successful when that sequence was replaced by mask tokens.

How AI was used

Five released structure-prediction models were run as inference engines over a curated test set of protein-peptide complexes, with no model fitted or fine-tuned in this study. Predictions were made from sequence with unpaired multiple-sequence alignments and no templates, taking the top-ranked of five predictions, and compared to the experimental structures using DockQ, backbone and all-atom RMSD, native-contact recovery, TM-score and DSSP-assigned secondary structure, alongside the models' own confidence outputs. The same models were then re-run under systematically altered inputs: no alignments, deeper paired alignments built from UniProt parent sequences with 50 or 100 residues of flanking context, those paired alignments with protein-peptide pairings shuffled, peptide alignments removed, native structures supplied as templates in place of alignments, peptide sequences replaced by unknown tokens, poly-glycine or scrambled residues, and Boltz-1 pocket residues or a Chai-1 distance restraint supplied. For AlphaFold2-Multimer, row- and column-wise self-attention between rows and columns belonging to different chains was masked inside the Evoformer, and masked-peptide predictions were read out from the distogram head using a pairwise-distance comparison metric. Model behaviour was related to training-set overlap by searching for homologous proteins by sequence or structure among structures released before each model's training cutoff and checking whether they bound a structurally similar partner, and to alignment statistics via inter-chain mutual information, Jensen-Shannon conservation and interface hydrophobicity.