Bias in artificial intelligence (AI), especially in healthcare, has significant consequences for equity, patient and provider trust, and patient outcomes. On 12 May 2025, the IHF Women in Leadership Special Interest Group held a learning session to explore how hospital leaders can reduce these biases.

Expert Dr Nasibeh Zanjirani Farahani from Mayo Clinic, US, shared insights into how and why bias enters AI systems, how it affects patient care and decision-making, and what leaders can do to create more fair and effective AI tools. Dr Moein Enayati, also from Mayo Clinic, then did a deep-dive into a use-case of a successful AI tool used at Mayo Clinic to diagnose hypertrophic cardiomyopathy (HCM). He provided real-world examples and shared practical insights and technical strategies to reduce bias during the training and evaluation of AI models.

What is bias in AI?

Bias in AI leads to unfair outcomes across patient subgroups, amplifying disparities based on race, gender, geography, and socioeconomic status. There are three main types of bias:

• Data bias stemming from unrepresentative or incomplete datasets.
• Algorithmic bias, when model design or selection amplifies disparities.
• Presentation bias, when information is displayed or interpreted unevenly.

To explain this, Dr Farahani used a fictional example of a robot named Doctor Bear Care. He was trained on images of sick boy children and therefore couldn’t recognize the symptoms of sick girl children.  The story showed clearly how bias can start with the data AI tools are trained on and then grow worse through model design and interpretation.

How to reduce bias across the AI lifecycle?

Bias can happen at any stage of an AI product’s development and use. Dr Farahani shared four key areas where concrete actions can make a difference:

• Data collection: Ensure representative data across race, gender, geography, and income levels.
• Model development: Include diverse perspectives in development teams and apply fairness metrics during training and validation.
• Deployment: Calibrate AI locally to the hospital context and provide training to end-users.
• Feedback: Engage users and communities to report problems and improve models. Involve clinical governance to respond to unintended consequences.

How to make AI fairer?

Healthcare leaders and decision-makers have a big role to play in ensuring AI products are as fair as possible. Even without technical expertise, they can:

• Champion inclusive data.
• Form cross-functional teams.
• Establish bias detection protocols.
• Monitor subgroup performance.
• Train users to understand and critically evaluate AI output.
• Promote patient-centred design and transparency.
• Collaborate externally to align with ethical standards.

Case in focus: AI for diagnosing HCM at Mayo Clinic

Dr Moein Enayati, Assistant Professor of Healthcare Systems Engineering at Mayo Clinic shared his experiences in developing AI to support diagnosis of hypertrophic cardiomyopathy (HCM), a rare heart condition. He started by sharing the general steps of developing any machine learning model for clinical decision making support including model development, implementation, deployment and evaluation. Key challenges included:

• Imbalanced data: limited number of HCM-positive cases.
• Heterogeneity across sites in terms of patient demographics distribution and standard operating procedures.
• Inconsistencies in measurement protocols, impacting consistency and generalization.
• Model bias, caused by inherent algorithmic preferences for certain feature types or statistical properties.

To address these issues, the team used advanced techniques like SMOTE (a resampling method), trained models on data from multiple sites, and focused on shared, standard measurements. They also ensured the model explained its results clearly to doctors and used thresholds that made sense clinically, not just statistically.

Hospital leaders must recognize and address AI bias. That means understanding where bias comes from, supporting diverse and collaborative teams, and keeping patients at the centre of the process. With the right oversight, training, and tools, AI has the potential to reduce health disparities, not deepen them.

Resources and tools

For leaders and developers looking to build or evaluate fair AI:

CHAI Applied Model Card: A standardized tool to evaluate AI models.

Bio-Clinical Bert: A language model tailored for biomedical data.

Echocardiographic diagnosis of hypertrophic cardiomyopathy by machine learning: Mayo Clinic’s published work on AI for echocardiographic diagnosis of HCM.