New Machine Learning Model Superior for Predicting Risk for In-Stent Restenosis in STEMI

The use of a new open-source machine learning model immediately following stent implantation in patients with ST-segment elevation myocardial infarction (STEMI) was found to provide a more accurate estimate of the risk for in-stent restenosis (ISR) compared with currently used parameters, according to an editorial published in the Canadian Journal of Cardiology.¹

In an article published in the same issue, investigators used several machine-learning approaches to create a risk model for predicting ISR following the placement of a coronary stent and revascularization.² Previously developed models (Prevention of Restenosis With Tranilast and Its Outcomes [PRESTO] and the Evaluation of Drug-Eluting Stents and Ischemic Events [EVENT] risk score) were found to have low predictive powers with areas under the receiver operating characteristic curves (AUC) ranging between 0.63 to 0.68.

A total of 263 patients with STEMI undergoing percutaneous coronary intervention (PCI) were recruited for a 2-by-2 randomized, open-label, multicenter trial in which paclitaxel-eluting and bare-metal stents were compared. Of the 263 patients, 23 had an ISR and 240 did not. Due to the small number of patients who had an ISR, the study authors acknowledged that these data were not ideal as input for a machine learning approach.

To address this, the investigators implemented a k-fold cross-validation which was meant to limit overfitting by repeatedly splitting and sampling the data. Due to the imbalance between cases and controls, the area under the precision-recall curves (AUC-PR) was used. This measure is less vulnerable to imbalanced datasets than AUC. The model thus-developed uses an extremely randomized trees (ERT) classifier.

Comparing AUC-PR values between the newly developed and previously published models, the investigators observed superior performance with the new model (ERT: AUC-PR, 0.46; PRESTO-1: AUC-PR, 0.31; PRESTO-2: AUC-PR, 0.27; EVENT: AUC-PR, 0.18).

Although future studies are needed with larger sample sizes to confirm the validity of this ERT model, the editorial authors noted that the feature selection approach may be useful for identifying novel predictors of ISR among other patient populations using the open-source code made available by the study authors. In addition, as the strongest predictors of ISR identified by the ERT model were readily available clinical factors, this model may easily be applied for the identification of the subset of patients with STEMI that are most likely to have an ISR.

“This study provides an important proof-of-concept demonstration that machine-learning models can be used to develop effective risk-prediction models for ISR. Other risk-prediction efforts that use similarly small or imbalanced datasets might consider adopting a similar approach of cross-validation and examination of AUC-PRs,” noted the editorial authors.

Disclosure: An author declared affiliations with industry. Please refer to the original editorial for a full list of disclosures.

References

1.  Avram R, Olgin J E and Tison G H. The rise of open-sourced machine learning in small and imbalanced datasets: predicting in-stent restenosis. Can J Cardiol. 2020;S0828-282X(20)30074-X. doi:10.1016/j.cjca.2020.02.002

2.  Sampedro-Gómez J, Dorado-Díaz P I, Vicente-Palacios V, et al. Machine learning to predict stent restenosis based on daily demographic, clinical, and angiographic characteristics. Can J Cardiol. 2020;S0828-282X(20)30072-6. doi:10.1016/j.cjca.2020.01.027