Evaluating the agreement and robustness of explainable AI in medical image recognition under natural and adversarial data corruption

The integration of Explainable AI (XAI) in healthcare improves the transparency and interpretability of machine learning models, allowing clinicians to understand model predictions and make well-informed medical decisions. However, the reliability of XAI methods remains uncertain, as explanations can vary significantly due to perturbations in input data. As a result, clinical applications become particularly vulnerable to misinterpretation, potentially affecting patient outcomes. Therefore, given the significance of XAI in healthcare and its influence on medical decisions, this thesis evaluates its robustness against data corruption and its agreement degree. Specifically, we systematically assess the stability of XAI techniques against both natural noise (e.g., compression artifacts) and adversarial manipulations, which can alter or distort XAI results and we evaluate how much different XAI techniques agree on the explanation given. Our approach involves defining a comprehensive set of evaluation metrics to quantify explanation consistency and proposing a novel evaluation protocol to analyze the resilience of different XAI methods under varying noise conditions. Through extensive experiments on three medical imaging datasets, we identify critical vulnerabilities in current XAI techniques, revealing inconsistencies in their robustness and agreement degree. While some methods maintain spatial stability, others preserve feature-wise consistency, leading to varying assessments of explanation reliability.