A shared test set for predicting which enzyme carries out a reaction

What this research was about

Enzymes are proteins that speed up chemical reactions in living things. Biologists label what an enzyme does with an enzyme commission number, or EC number, a four-part code that narrows from a broad class of chemistry down to a specific reaction. The trouble is that most proteins found by sequencing have never been tested in a laboratory, so their EC number is a guess. The usual way to guess is to look for a known protein with a similar sequence of amino acids. That works when a close relative is already labelled, and gets shaky when the nearest known protein is only distantly similar, or when one enzyme handles several different reactions.

The authors built CARE, a collection of curated data and test splits for two questions. The first is: given a protein sequence, what is its EC number? The second runs the other way: given a chemical reaction, which EC number of enzyme performs it? The protein set holds 185,995 sequence-EC pairs and the reaction set 61,766 reaction-EC pairs, covering 4,960 EC numbers. The test splits are designed to be awkward on purpose, holding back sequences that share under 30 per cent and 30 to 50 per cent of their sequence with the training data, plus sequences that earlier work had labelled wrongly and enzymes that do more than one job.

Where AI came in

The reported results are the scores of machine-learning models on these splits, so they exist only by running those models. The authors retrained three existing models, CLEAN, Pika and CLIPZyme, on their training data and asked them for the EC number of each held-out sequence or reaction. They also prompted two off-the-shelf systems, ChatGPT and ChemCrow, directly through their programming interfaces. And they introduced CREEP, which was made by fine-tuning two pretrained models, rxnfp for reactions and ProtT5 for protein sequences, so that a reaction and the proteins that carry it out land near each other in a shared numerical space. A version of CREEP adds written descriptions of EC numbers, encoded by a third model, SciBERT.

AI also filled a gap in the comparison baselines. One standard non-learned method searches for proteins of similar three-dimensional shape rather than similar sequence, which needs a structure for every protein. Where no structure existed in the AlphaFold database, ESMFold predicted one from the sequence, standing in for experimental structure determination. Those structures fed the Foldseek shape-search baseline, which was scored alongside random ordering, a sequence search with BLASTp and a chemical-similarity measure of reactions. On classification, the paper reports CLEAN doing best for distantly related sequences, with the sequence and shape searches close behind and shape search best on the previously misclassified split. On the harder retrieval splits, CREEP with text beat the chemical-similarity baseline, though accuracies there stayed low for every method.

Written by AIxSci from the checked record below, to give context for readers outside the field. It is not part of the record.