I am excited to share that one of our recent papers written in collaboration with EMD Serono, Pharmacometric modeling and machine learning analyses of prognostic and predictive factors in the JAVELIN Gastric 100 phase III trial of avelumab, has been recognized as one of the most cited in its publication last year. Reflecting on this achievement, I wanted to provide some insights into why I believe this paper resonated so strongly with the pharmacometrics community and became such a valuable resource.
Exploring the Intersection of Machine Learning and Pharmacometric Modeling
Our paper took a comprehensive approach to integrating machine learning into traditional pharmacometric workflows. We employed methods such as random forests, SIDEScreen, and variable-importance assessments to identify prognostic and predictive factors associated with long-term OS and tumor growth dynamics (TGDs). In doing so, we not only confirmed previous traditional analyses but also provided a detailed counter-example to illustrate the value of combining empirical ML methods with expert knowledge . This realistic comparison of methodologies was crucial to the validation of new approaches against established ones.
The Importance of Negative Analyses
Another significant aspect of our paper was the emphasis on sharing negative analyses. Too often, negative results are not published, which creates a biased understanding of research outcomes. By documenting our findings that no novel effects were identified to explain the differences observed in the trial, we contributed to a more balanced scientific discourse. Our analysis confirmed that while machine learning offers valuable tools, it is not a panacea and must be applied judiciously.
Collaboration and Methodological Insights
The collaboration between MetrumRG's statistics and pharmacokinetics/ pharmacodynamics (PKPD) groups and EMD Sereno's PKPD group enriched the study.
It allowed us to take a closer look at the nuances of modeling tumor growth and integrating these models with overall survival data. We found that the heterogeneity in both tumor growth dynamics and overall survival modeling required terms without strong biological or mechanistic motivation to describe the data accurately. This highlights the power of combining expert knowledge with machine learning, as it enables the discovery of these unexpected relationships and offers a promising direction for future research.
Future Directions and Community Impact
Our paper provided a valuable workflow example, combining machine learning with traditional modeling approaches. This has proven useful for the community, as evidenced by the paper's citations. Moving forward, we hope that future research in pharmacometrics will continue to explore these methodologies, potentially finding more consistent signals in the data and improving the predictability of oncology outcomes.
We are proud to have contributed to this important dialogue and look forward to continuing our work in this exciting field.
Abstract and Discussion Highlights
For those interested in a deeper dive, here are the key points from the abstract and discussion of the paper:
Abstract:
Avelumab (anti–PD-L1) is an approved anticancer treatment for several indications. The JAVELIN Gastric 100 phase III trial did not meet its primary objective of demonstrating superior overall survival (OS) with avelumab maintenance versus continued chemotherapy in patients with advanced gastric cancer/gastroesophageal junction cancer; however, the OS rate was numerically higher with avelumab at timepoints after 12 months. Machine learning (random forests, SIDEScreen, and variable-importance assessments) was used to build models to identify prognostic/predictive factors associated with long-term OS and tumor growth dynamics (TGDs). Baseline, re-baseline, and longitudinal variables were evaluated as covariates in a parametric time-to-event model for OS and Gompertzian population model for TGD. The final OS model incorporated a treatment effect on the log-logistic shape parameter but did not identify a treatment effect on OS or TGD. Variables identified as prognostic for longer OS included older age; higher gamma-glutamyl transferase (GGT) or albumin; absence of peritoneal carcinomatosis; lower neutrophil-lymphocyte ratio, lactate dehydrogenase, or C-reactive protein (CRP); response to induction chemotherapy; and Eastern Cooperative Oncology Group performance status of 0. Among baseline and time-varying covariates, the largest effects were found for GGT and CRP, respectively. Liver metastasis at re-baseline predicted higher tumor growth. Tumor size after induction chemotherapy was associated with the number of metastatic sites and stable disease (vs. response). Asian region did not impact OS or TGD. Overall, an innovative workflow supporting pharmacometric modeling of OS and TGD was established. Consistent with the primary trial analysis, no treatment effect was identified. However, potential prognostic factors were identified.
Discussion:
This longitudinal pharmacometric analysis did not identify any significant treatment effects of avelumab versus chemotherapy in the maintenance treatment of advanced GC/GEJC, consistent with the primary analysis of the JAVELIN Gastric 100 trial. Disease models of OS and TGD were developed by integrating covariates efficiently informed by ML methods, and covariates potentially prognostic of OS and TGD were identified. However, no predictive factors associated with OS or TGD during avelumab treatment were found.
The analyses presented provide an example of incorporating ML approaches into a traditional pharmacometric workflow. Specifically, separate RF models were used to identify prognostic factors for OS and tumor size end points, which were then added to parametric TTE and population TGD models, respectively. Supplementing ML with parametric methods resulted in more-interpretable final models than the RF alone, particularly for noisy time-varying covariates and given the multistage trial design (induction and maintenance phases). Furthermore, in comparison to parametric modeling alone and/or using stepwise regression or hypothesis testing, the ML approach was faster and was performed using a single pass over the data. One potential limitation of this workflow arises in the translation of the nonlinear and interacting effects inherent in ML models into parametric forms. We started with linear effects and used diagnostic plots to guide refinement of the model. Alternative approaches could also be considered to guide the initial choice of covariate-effect relationships, such as using partial dependence or accumulated local effect plots.
In summary, the longitudinal models developed provide a quantitative framework that can be leveraged as a disease model for GC in the maintenance setting. While no subpopulation for which avelumab was superior to chemotherapy was identified, the identified potential prognostic factors require further confirmation and may inform future studies in this setting.
Matthew Wiens, M.A., Senior Scientist II
Matthew joined Metrum in 2019 as a Research Scientist. He holds an M.A. in Statistics from Boston University. Prior to Metrum, he worked for a variety of startup technology companies where he applied Bayesian methodologies in predictive models based on remote sensing data. His ongoing interests include communicating and leveraging uncertainty from a Bayesian perspective in scalable modeling and simulation projects.