The integration of Artificial Intelligence (AI) in medical imaging has the potential of revolutionizing healthcare, allowing professionals to analyze the details of the human body with unprecedented accuracy and speed. Despite the potential of AI to transform medical diagnostics, it faces significant challenges, primarily due to limited data sets and the difficulty in generalizing across diverse medical scenarios. Large datasets like the UK Biobank and TCGA provide a foundation, but generalization and reliability in diverse situations remain hurdles. Particularly challenging are Out-of-Distribution (OOD) shifts caused by demographic changes, advancements in imaging technologies, and evolution in clinical practices, which pose risks to the reliability and trustworthiness of AI systems. One of the critical issues is the variability in cancer subtypes, which, despite similar visual presentations, vary widely in prognosis. Training models to accurately differentiate these subtypes is hindered by data privacy laws and the scarcity of samples for rare subtypes. The COVID-19 pandemic further exemplified the shortcomings of current state-of-art in medical imaging AI. These scenarios underscore the need for AI systems that can adapt swiftly to new challenges, maintaining reliable support in critical diagnostic processes. This dissertation addresses these challenges by proposing methodologies to enhance the reliability of Deep Learning (DL)-based systems in medical imaging. Key approaches include Multi-headed Varational Inference (VIMH) for uncerainty estimation, Sliding-Window Optimal Transport for OOD Detection (SWOT), and Histopathology Artifact Restoration Pipeline (HARP). While VIMH offers improved robustness against OOD shifts, it demands specialized training. In contrast, SWOT and HARP offer post-hoc solutions applicable to existing AI models, enhancing diagnostic precision and ensuring reliability in the deployed AI systems. The dissertation also explores dynamic learning settings such as Continual Learning (CL) and Federated Learning (FL). In the federated histopathology settings, Federated Stain Normalization with BottleGAN (BottleGAN) presents an ideal solution for overcoming limited data annotation and data heterogeneity in Computational Pathology (CP). Furthermore, Closing-the-Loop with Radiologists (CtLwR) integrates transparency into the AI decision-making process and leverages structured report to enable CL. These approaches are vital in adapting to varying populations and institutional changes, maintaining the reliability of DL-based models. In conclusion, the medical imaging sector’s increasing use of AI requires a balanced approach that prioritizes patient privacy, reliability, and ethical standards. The shift from AI vs. clinicians to AI collaborating with clinicians signals a significant change, combining efficiency with patient-focused care. To successfully navigate the unknowns of the medical field, AI must address the challenges and ensure its integration into healthcare is beneficial and safe.

For IEEE JBHI article published: In reference to IEEE copyrighted material which is used with permission in this thesis, the IEEE does not endorse any of Technical University of Darmstadt’s products or services. Internal or personal use of this material is permitted. If interested in reprinting/republishing IEEE copyrighted material for advertising or promotional purposes or for creating new collective works for resale or redistribution, please go to http://www.ieee.org/publications_standards/publications/rights/rights_link.html to learn how to obtain a License from RightsLink. If applicable, University Microfilms and/or ProQuest Library, or the Archives of Canada may supply single copies of the dissertation.