Predicting how cells respond to genetic modifications or chemical treatments is a cornerstone of drug discovery. If we can accurately simulate the impact of a gene knockout or a new drug candidate, we can identify therapeutic targets more quickly and flag potential side effects much earlier in the development process.
The rise of biological Foundation Models (FMs) has sparked a significant debate in perturbation modeling. While some researchers claim these models provide a leap forward, others argue they perform no better than simple, decades-old statistical baselines.
To resolve this, we conducted the most extensive analysis to date, evaluating over 600 different model variants. We tested these models across diverse prediction tasks, looking at various architectures (like Transformers and GNNs) and data sources (including DNA, protein sequences, and gene expression).
We developed new methods to integrate multiple FMs using an attention-based “fusion” approach. By combining diverse biological perspectives, these integrated models outperformed any individual FM.
Key findings include:
- Data modality is key. The most significant determinant of success is the data modality used for training the FM. Models based on the interactome (molecular interaction networks) consistently outperformed baselines, while those based strictly on DNA or protein sequences often underperformed
- Approaching fundamental limits. Our fusion-based model reached the performance ceiling. In the Jurkat and K562 cell lines, our results matched the estimated experimental error limit (i.e. the best achievable performance for any prediction).
- Chemical perturbations are harder. Predicting chemical perturbations remains more difficult due to the massive search space of small molecules and a relative lack of interaction data. However, FMs still provided significant improvements over baselines in specific formulations.
While much work remains, these results indicate that given enough data and the right modality, Foundation Models can lead to highly accurate, actionable simulations of cellular behavior.
For more details, please see our preprint on bioRxiv and its accompanying GitHub repository.
