Machine Learning-Optimized EHR Audit Improves Heart Failure Classification

Heart failure (HF) diagnoses in primary care electronic health records (EHRs) are inaccurate, but machine learning optimization may help reduce HF misclassification, according to study results published in ESC Heart Failure.

For the current study, researchers performed a secondary analysis of the OSCAR-HF study, which consisted of 51 general practitioners (GPs), who were asked to classify patients with either HF or nonHF from an extended audit. The query-based audit mapped known HF risk factors, signs, symptoms, and medications from the GPs’ EHRs. Researchers compared registered HF diagnoses before and after GPs’ review. To examine audit performance, researchers used GPs’ HF assessment as a primary outcome and audit queries as dichotomous predictor variables for a logistic regression model and gradient-boosted machine (GBM) decision tree algorithm.

Investigators identified 4678 patients with possible HF using the query-based audit. Out of 310 patients with registered HF before GP assessment, 146 (47.1%) were ruled to not have HF after their GP assessment, indicating over-registration. Out of 538 patients with registered with HF after GP assessment, 374 (69.5%) did not have registered HF prior to their GP assessment, indicating under-registration.

The predictive performances of the GBM model (area under the curve [AUC] 0.70; 95% CI, 0.65-0.77) and the logistic regression model (AUC 0.69; 95% CI, 0.64-0.75) were comparable. Reducing the set of predictor variables to the 10 most important did not significantly impact the performance of the GBM model. The optimized query set enabled the identification of 461 of 538 (85.7%) patients with HF and could reduce GPs’ screening caseload by 33% (in which 1537 of 4678 patients could be ruled out in the audit).

Limitations included the use of GP assessment as the outcome of interest rather than those presented by the validation panel. In addition, the variables were only tested on the patients identified with possible HF and not in the broader primary care population.

“Diagnostic coding of HF in primary care health records is inaccurate with a high degree of underclassification and overclassification,” the study authors noted. “An optimized query set using only search strings for cardiomyopathy, ischemic heart disease, atrial fibrillation, digoxin, mineralocorticoid receptor antagonists, and combinations of RAS inhibitors and beta-blockers with diuretics enabled identification of more than 80% of GPs’ self-assessed HF population, albeit with modest specificity.”

Reference

Raat W, Smeets M, Henrard S, et al. Machine learning optimization of an electronic health record audit for heart failure in primary care. ESC Heart Fail. Published online November 23, 2021. doi:10.1002/ehf2.13724