A. Kia, P. Timsina
Mount Sinai Health System,
United States
Keywords: multi-modality fusion framework, deep learning, clinical decision support, real-time
Summary:
Synopsis Every second, modern health care ecosystem is generating a wide range of clinical data format including structured, unstructured, image, and high-frequency waveform data. The volume of data is also growing at an exponential rate. On the other hand, a paradigm shift is happening in medicine that makes clinical interventions more personalized, prognostic and data driven. Clinical notes, pathology reports, and radiology impressions are increasingly available by the extensive use of EHR platforms in different clinical settings. On the other side, there are many clinical information such as cancer staging, cancer prognosis, and psychiatry observations can be found exclusively in Medical Imaging and reports. To address these challenges, we developed and deployed real-time multi-modality pipelines to ingest different clinical modalities including documentations/notes, Medical Images, and structures/semi structured data, process them and generate different risk stratification scores for wide range of clinical service lines from critical care medicine to psychiatry in outpatient and inpatient settings. In this talk, we will share the computational design and workflows, the transfer learning approaches, and the operationalizing methods being used to incorporate the machine learning clinical decision support (CDS) tools with the clinical workflows. Key Takeaways • The rationale behind of the need for a real-time multi-modality CDS tool in personalized medicine • The architecture of the multi-modality machine learning platforms including components and the way they should talk to each other • Highlight the challenges and the lessons learned from developing and deploying process and our approach in building capacity for knowledge sharing and collective learning in an acute care setting Target Audience The talk is primarily relevant to Data Scientists, Data Engineers, Chief Data Officers, Chief Information Officers, and Chief Innovation Officers. The audience should have preliminary knowledge of No-SQL, Machine Learning, and NLP Applications.