Abstract:Introduction Developments in Artificial Intelligence (AI) are adopted widely in healthcare. However, the introduction and use of AI may come with biases and disparities, resulting in concerns about healthcare access and outcomes for underrepresented indigenous populations. In New Zealand, Māori experience significant inequities in health compared to the non-Indigenous population. This research explores equity concepts and fairness measures concerning AI for healthcare in New Zealand. Methods This research considers data and model bias in NZ-based electronic health records (EHRs). Two very distinct NZ datasets are used in this research, one obtained from one hospital and another from multiple GP practices, where clinicians obtain both datasets. To ensure research equality and fair inclusion of Māori, we combine expertise in Artificial Intelligence (AI), New Zealand clinical context, and te ao Māori. The mitigation of inequity needs to be addressed in data collection, model development, and model deployment. In this paper, we analyze data and algorithmic bias concerning data collection and model development, training and testing using health data collected by experts. We use fairness measures such as disparate impact scores, equal opportunities and equalized odds to analyze tabular data. Furthermore, token frequencies, statistical significance testing and fairness measures for word embeddings, such as WEAT and WEFE frameworks, are used to analyze bias in free-form medical text. The AI model predictions are also explained using SHAP and LIME. Results This research analyzed fairness metrics for NZ EHRs while considering data and algorithmic bias. We show evidence of bias due to the changes made in algorithmic design. Furthermore, we observe unintentional bias due to the underlying pre-trained models used to represent text data. This research addresses some vital issues while opening up the need and opportunity for future research. Discussions This research takes early steps toward developing a model of socially responsible and fair AI for New Zealand's population. We provided an overview of reproducible concepts that can be adopted toward any NZ population data. Furthermore, we discuss the gaps and future research avenues that will enable more focused development of fairness measures suitable for the New Zealand population's needs and social structure. One of the primary focuses of this research was ensuring fair inclusions. As such, we combine expertise in AI, clinical knowledge, and the representation of indigenous populations. This inclusion of experts will be vital moving forward, proving a stepping stone toward the integration of AI for better outcomes in healthcare.

Young people and other vulnerable groups are overlooked in AI algorithm development

Applied artificial intelligence for global child health: Addressing biases and barriers

Bias in artificial intelligence algorithms and recommendations for mitigation

Sources of bias in artificial intelligence that perpetuate healthcare disparities—A global review

The Potential of Diverse Youth as Stakeholders in Identifying and Mitigating Algorithmic Bias for a Future of Fairer AI

Engaging children and young people on the potential role of artificial intelligence in medicine

Data and model bias in artificial intelligence for healthcare applications in New Zealand

Perceptions and attitudes of Children and Young People to Artificial Intelligence in Medicine

Practical, epistemic and normative implications of algorithmic bias in healthcare artificial intelligence: a qualitative study of multidisciplinary expert perspectives

Digital Ageism: Challenges and Opportunities in Artificial Intelligence for Older Adults

Mitigating the risk of artificial intelligence bias in cardiovascular care

Age-related bias and artificial intelligence: a scoping review

AI revolution in healthcare and medicine and the (re-)emergence of inequalities and disadvantages for ageing population

New Horizons in artificial intelligence in the healthcare of older people

A translational perspective towards clinical AI fairness

The bias algorithm: how AI in healthcare exacerbates ethnic and racial disparities – a scoping review

Bias and Non-Diversity of Big Data in Artificial Intelligence: Focus on Retinal Diseases

Oxford Handbook on AI Ethics Book Chapter on Race and Gender

Can AI Help Reduce Disparities in General Medical and Mental Health Care?

Addressing Bias in Artificial Intelligence in Health Care

Bias in data‐driven artificial intelligence systems—An introductory survey