Machine Learning for Healthcare 2024
The potential for artificial intelligence (AI) to benefit our health must be balanced against the risks posed by algorithmic bias and harms. These technologies may work better for some groups and worse for others, causing or worsening health inequalities.
STANDING Together aims to ensure that inclusivity and diversity are considered when developing health datasets and AI health technologies.
We have built recommendations, through an international consensus process, which provide guidance on transparency around 'who' is represented in the data, 'how' people are represented, and how data is used when developing AI technologies for healthcare.
Click here to learn more about the STANDING Together recommendations
ML4HC dataset poster final print copy.pdfReferences for our poster at Machine Learning for Healthcare:
Obermeyer, Ziad, Brian Powers, Christine Vogeli, and Sendhil Mullainathan. ‘Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations’. Science 366, no. 6464 (25 October 2019): 447–53. https://doi.org/10.1126/science.aax2342.
Wong, Andrew, Erkin Otles, John P. Donnelly, Andrew Krumm, Jeffrey McCullough, Olivia DeTroyer-Cooley, Justin Pestrue, et al. ‘External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients’. JAMA Internal Medicine 181, no. 8 (1 August 2021): 1065–70. https://doi.org/10.1001/jamainternmed.2021.2626.
Cao, Jie, Xiaosong Zhang, Vahakn Shahinian, Huiying Yin, Diane Steffick, Rajiv Saran, Susan Crowley, et al. ‘Generalizability of an Acute Kidney Injury Prediction Model across Health Systems’. Nature Machine Intelligence 4, no. 12 (December 2022): 1121–29. https://doi.org/10.1038/s42256-022-00563-8
Seyyed-Kalantari, Laleh, Haoran Zhang, Matthew B. A. McDermott, Irene Y. Chen, and Marzyeh Ghassemi. ‘Underdiagnosis Bias of Artificial Intelligence Algorithms Applied to Chest Radiographs in Under-Served Patient Populations’. Nature Medicine 27, no. 12 (December 2021): 2176–82. https://doi.org/10.1038/s41591-021-01595-0.
Chen, Irene Y., Emma Pierson, Sherri Rose, Shalmali Joshi, Kadija Ferryman, and Marzyeh Ghassemi. ‘Ethical Machine Learning in Healthcare’. Annual Review of Biomedical Data Science 4 (July 2021): 123–44. https://doi.org/10.1146/annurev-biodatasci-092820-114757.
Khan, Saad M., Xiaoxuan Liu, Siddharth Nath, Edward Korot, Livia Faes, Siegfried K. Wagner, Pearse A. Keane, Neil J. Sebire, Matthew J. Burton, and Alastair K. Denniston. ‘A Global Review of Publicly Available Datasets for Ophthalmological Imaging: Barriers to Access, Usability, and Generalisability’. The Lancet Digital Health 3, no. 1 (1 January 2021): e51–66. https://doi.org/10.1016/S2589-7500(20)30240-5.
Wen, David, Saad M. Khan, Antonio Ji Xu, Hussein Ibrahim, Luke Smith, Jose Caballero, Luis Zepeda, et al. ‘Characteristics of Publicly Available Skin Cancer Image Datasets: A Systematic Review’. The Lancet Digital Health 4, no. 1 (1 January 2022): e64–74. https://doi.org/10.1016/S2589-7500(21)00252-1.
Wu, Jiageng, Xiaocong Liu, Minghui Li, Wanxin Li, Zichang Su, Shixu Lin, Lucas Garay, et al. ‘Clinical Text Datasets for Medical Artificial Intelligence and Large Language Models — A Systematic Review’. NEJM AI 1, no. 6 (23 May 2024): AIra2400012. https://doi.org/10.1056/AIra2400012.
Liu, Xiaoxuan, Joseph Alderman, and Elinor Laws. ‘A Global Health Data Divide’. NEJM AI 1, no. 6 (23 May 2024): AIe2400388. https://doi.org/10.1056/AIe2400388.
Alderman JE, Charalambides M, Sachdeva G, Laws E, Palmer J, Lee E et al. Revealing transparency gaps in publicly available Covid-19 datasets used for medical artificial intelligence development: a systematic review. (in press). https://research.birmingham.ac.uk/en/publications/revealing-transparency-gaps-in-publicly-available-covid-19-datase
Ibrahim, Hussein, Xiaoxuan Liu, Nevine Zariffa, Andrew D. Morris, and Alastair K. Denniston. ‘Health Data Poverty: An Assailable Barrier to Equitable Digital Health Care’. The Lancet Digital Health 3, no. 4 (1 April 2021): e260–65. https://doi.org/10.1016/S2589-7500(20)30317-4.
The International Skin Imaging Collaboration (online). https://www.isic-archive.com/
Kermany, Daniel S., Michael Goldbaum, Wenjia Cai, Carolina C. S. Valentim, Huiying Liang, Sally L. Baxter, Alex McKeown, et al. ‘Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning’. Cell 172, no. 5 (22 February 2018): 1122-1131.e9. https://doi.org/10.1016/j.cell.2018.02.010.
Crenshaw, Kimberle. ‘Demarginalizing the Intersection of Race and Sex: A Black Feminist Critique of An tidiscrimination Doctrine, Feminist Theory and Antiracist Politics’. University of Chicago Legal Forum, 1989. https://chicagounbound.uchicago.edu/uclf/vol1989/iss1/8.
Gichoya, Judy Wawira, Imon Banerjee, Ananth Reddy Bhimireddy, John L Burns, Leo Anthony Celi, Li-Ching Chen, Ramon Correa, et al. ‘AI Recognition of Patient Race in Medical Imaging: A Modelling Study’. The Lancet. Digital Health 4, no. 6 (June 2022): e406–14. https://doi.org/10.1016/S2589-7500(22)00063-2.
STANDING Together is building STANdards for data Diversity, INclusivity, & Generalisablity. Established in 2021 as part of the NHS AI Lab’s AI Ethics initiative, it is a partnership between over 30 academic, regulatory, policy, industry, and charitable organisations worldwide. STANDING Together is funded by the NHS AI Lab at the NHS Transformation Directorate and The Health Foundation and managed by the National Institute for Health and Care Research (AI_HI200014).