Assessment of Intelligent Algorithms for Detection, Diagnosis and Risk Stratification of Tumors of the Kidney and Prostate

The COMFORT study aims to evaluate an AI-based platform developed to support the clinical assessment of patients with kidney and prostate cancer. The COMFORT platform provides AI-generated outputs intended to assist clinicians in the assessment and characterization of suspected lesions. The study evaluates not only the technical performance of the AI algorithms but also their integration into clinical practice, their fairness across different groups, and potential privacy risks. The pipelines developed for kidney and prostate cancer provide outputs relevant to the respective diagnostic pathways. Depending on the clinical application, these outputs include lesion stratification and assessments such as PI-RADS, Bosniak, and RENAL scores. The performance of the AI algorithms will be evaluated by comparing model-generated predictions with the corresponding clinical reference standards. The evaluation will include measures of classification performance and agreement between AI-generated assessments and the reference standard. Performance will also be evaluated separately for the relevant cancer types. An additional component of the study concerns the fairness of the COMFORT AI models. Model performance will be examined across different participant groups to identify potential differences in the behavior of the algorithms between groups. This analysis will complement the overall evaluation of model performance and contribute to the assessment of the reliability of the platform across the study population. The study will also evaluate the integration of the COMFORT platform into clinical practice from the perspective of participating clinicians. Questionnaires and structured interviews will be used to assess clinicians' perceptions and experiences with the platform. Areas of evaluation include acceptability, usability, trustworthiness of AI-generated outputs, perceived information integration, workload management, perceived effectiveness, care management efficiency, and AI literacy. These assessments will provide information on how clinicians interact with the platform and perceive its potential contribution to clinical practice. In addition to these subjective assessments, the effectiveness of the COMFORT platform in assisting clinicians during clinical reading will be evaluated objectively. AI-assisted clinical reading will be compared with manual clinical reading in terms of the time required to complete the reading and the quality of the resulting clinical assessment. These evaluations will be considered at individual study sites as well as across the study to examine the implementation of the platform in different clinical settings. Potential privacy risks associated with the COMFORT AI models will also be investigated. A privacy causal analysis approach will be used to examine potential causes of privacy leakage associated with AI model outputs and to characterize their potential causal effects. Overall, the study is designed to provide a comprehensive evaluation of the COMFORT platform across several complementary dimensions, including AI model performance, fairness, clinical usability and acceptability, effects on clinical reading, and potential privacy risks. This combined evaluation is intended to provide evidence regarding the performance of the platform and its use in the clinical assessment of kidney and prostate cancer.