(M0930-05-29) Benchmarking Large Multi-task Machine Learning Models for ADME, Toxicology and Drug Discovery Endpoints

Purpose: Computation of absorption, distribution, metabolism, excretion, toxicity (ADME/Tox) properties and drug discovery endpoints using machine learning and exploiting statistical techniques can focus and prioritize the use of expensive assays very early in the drug discovery process and minimize late-stage failures of potent lead compounds. Some endpoints, however, have little data available, and the training of a typical ligand-based machine learning model is therefore challenging. Recently, large multi-task networks have been trained using transfer-learning, in which a related task is trained alongside the primary task(s), with the model then gaining predictive performance on the primary task.

Methods: To determine if transfer learning can enhance the predictive power sparse-data for various endpoints, we decided to take two approaches: 1) determine if predicting multiple endpoints at once will benefit from transfer learning, and 2) take advantage of the large number of datasets that we have curated and combine them into one large training set with many endpoints. As an example we have built a deep learning Convolutional Long Short Term Memory (ConvLSTM) model as well as fine-tuned a pre-trained Transformer model (MolBART) on additional datasets (e.g. 42 toxicity datasets). We then created an aggregate dataset with 2400 targets from an extract of from ChEMBL to determine whether the toxicology target molecules with a small number of data points (less than 250 datapoints) would benefit from transfer learning from the initial pre-trained molBART model on many different targets. To do this, we compared against the gold standard Support Vector Regression (SVR) model.

Results: The fine-tuned transformer model significantly outperformed the ConvLSTM model on these 42 toxicity datasets, with an average R² of 0.53 (range: 0.31 - 0.77) for the MolBART model vs. an average R² of 0.46 (range: 0 - 0.74) for the Conv-LSTM model. 19/42 targets had an R² delta 0.05 in favor of MolBART for the 42 toxicity datasets. Prediction measurements showed that we were able to improve the accuracy of the toxicology predictions for molecules with a small number of data points using MolBART. The average R²improvement of MolBART was 0.33 when datasets were small, while SVR only increased R² 0.16 in larger datasets.

Conclusion: The benefit of using transformer models becomes less significant as the dataset size increases, as there is then sufficient information in the dataset for the model to make predictions without the benefit of transfer learning. We will describe our benchmarking with additional ADME and drug discovery datasets and indicate how these models may aid our efforts in generative drug discovery.

Acknowledgements:SE is owner and SS and FU are employees of Collaborations Pharmaceuticals, Inc.