Welcome to the final post of a three-part blog series reflecting on the ACoP (American Conference on Pharmacometrics) 13 meeting held October 30th - November 2nd, 2022.
In the previous (second) post of the series we explored machine learning (ML) methodologies, specifically, Shapley values to aid in interpreting ML models. In the first post we discussed the realistic goals, reasonable scope, and perspective on the effort required to integrate artificial intelligence/machine learning (AI/ML) in pharmacometrics. Today, we dive deeper into several specific applications of large-scale ML projects providing additional insights to pharmacometric projects.
Pharmacometrics-specific ML approaches for classically challenging models
In some modeling projects, parametric models fail to adequately capture data, and show lack of fit in visual predictive checks (VPCs) and/or do not provide sufficient value to answer the question of interest. In these cases, ML models have provided new perspectives and approaches to pharmacometric workflows. Neural ordinary differential equation architecture (TDNODE) from James Lu and Nicholas Ellinwood (Explainable Machine Learning for Disease Progression Modeling & Digital Twins. Session 3A.) was used to predict overall survival (OS) from tumor growth dynamics (TGD). A key insight was that the parametric TGD model was primarily a data reduction exercise for OS modeling because a patient’s tumor size trajectory is summarized in (usually) 3 parameters from the parametric model. Instead, in the ML methodology, an autoencoder summarizes the longitudinal data for input to the OS model, which in this example, was more powerful and better captured important features in the data that were subsequently used for predicting OS.
My experience is that parametric models doing the same thing have only modest performance, and the value of the parameters for the TGD model is modest compared to other covariates for predicting OS. This was also a case where the ML model provided substantial improvements over the parametric model! A common anecdote at MetrumRG and in the statistics community is the experience of a logistic regression and a complex ML model providing equal performance. Another reason I thought this example was noteworthy was because it leveraged neural net architecture to better answer a scientific question rather than the idea of putting all the available data into a generic neural net architecture and letting ML figure it out. Furthermore, it moved beyond the “predict Y from X a bit better” paradigm. However, this example also made clear the effort that this model required, and is one compelling reason why ML isn't ready for routine application, but already can provide value in targeted applications.
It’s exciting to see the progress of ML in pharmacometrics, the practical success teams are having using ML methods, and the interdisciplinary collaboration that is being fostered.
Thank you to everyone who I had the opportunity to engage with at ACoP and those whom I presented to. At MetrumRG, we’re looking forward to continuing these discussions, and seeing what exciting developments there are between now and the next ACoP. In the meantime, stay up to date with our team by subscribing to our newsletter, and check out all our posters and publications from ACoP: https://www.metrumrg.com/publications/
What did you take away from ACoP, if you attended? Comment below!