A Hybrid AI Framework to Address the Issue of Frequent Missing Values with Application in EHR Systems: the Case of Parkinson’s Disease

Abstract

Electronic health record (EHR) systems hold vast amounts of patient data that, when analyzed with explainable AI techniques and predictive analytics, can improve clinical decision support systems (CDSS). However, the volume of data, with millions of patient records and hundreds of features collected over time, presents significant challenges, including handling missing values. In this project, we introduce a framework that addresses the issue of incompleteness in EHR data, enabling researchers to select the most important variables at an acceptable level of missing data to develop accurate predictive models. We demonstrate the effectiveness of this framework by applying it to developing a CDSS for detecting Parkinson's disease based on large EHR data. Parkinson's disease is hard to diagnose, and even specialists' diagnoses can be inaccurate; moreover, limited access to specialists in remote areas results in many undiagnosed patients. Our framework can be integrated into EHR systems or used as an independent tool by healthcare practitioners who are not necessarily specialists, bridging the gap in specialized care in remote areas. Our results show that the framework improves the accuracy of predictive models and identifies patients with Parkinson's disease who might otherwise go undiagnosed.