Our Work
Work Packages
1
Defining best practice for dataset curation and use
Scoping review of existing dataset standards
Draft items for multi-stakeholder eDelphi (click HERE to find out more)
2
Mapping dataset deficiencies in priority disease areas
Identify datasets used in priority disease areas: COVID-19, breast cancer, heart failure
Assess datasets using proposed standards
3
Identifying and overcoming barriers to curating datasets
Conduct semi-structured interviews with dataset curators/users about implementing the standards
Publications
The STANDING Together Working Group, Nature Medicine
To launch the first round of the STANDING Together Delphi Study, an announcement paper was published in Nature Medicine in September 2022.
The full paper can be found at: https://doi.org/10.1038/s41591-022-01987-w
The availability of health datasets has accelerated digital health research. Ophthalmology has been one of the leading areas of innovation, where several public datasets for ophthalmic imaging have been use in machine learning research. Datasets are a critical component for machine learning algorithm development, hence these need careful scrutiny prior to use. Prior to our review, it was previously unknown how many ophthalmic datasets existed, their degree of accessibility... Read more
Summary by Dr David Wen
Freely available (open access) datasets containing skin images are frequently used to develop deep learning algorithms for skin cancer diagnosis. As these algorithms are heavily influenced by the images that they are trained on, it is important that the composition and characteristics of datasets are outlined, such as which populations images are taken from. This information is often... Read more
STANDING Together: STANdards for Data Diversity, Inclusivity and Generalisability
Presented by Dr Joe Alderman at Machine Learning for Healthcare, 2022
Click here to view the poster
Click here to watch the presentation
Insights from the STANDING Together Consensus Study
Presented by Dr Elinor Laws at The Symposium on Artificial Intelligence for Learning Health Systems, 2023
Click here to view the poster
ML4HC poster reference list
Artificial Intelligence - how to get it right. NHSX. October 2019. Available from URL: https://transform.england.nhs.uk/media/documents/NHSX_AI_report.pdf
Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. The Lancet Digital Health. 2019 Oct 1;1(6):e271–97. Available online at: https://www.thelancet.com/journals/landig/article/PIIS2589-7500(19)30123-2/fulltext
Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019 Oct 25;366(6464):447–53. Available online at: https://www.science.org/doi/10.1126/science.aax2342
Seyyed-Kalantari L, Zhang H, McDermott MBA, Chen IY, Ghassemi M. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat Med. 2021 Dec;27(12):2176–82. Available online at: https://www.nature.com/articles/s41591-021-01595-01.
The health of people from ethnic minority groups in England [Internet]. The King’s Fund. 2021 [cited 2022 Apr 12]. Available from: https://www.kingsfund.org.uk/publications/health-people-ethnic-minority-groups-england1.
Khan SM, Liu X, Nath S, Korot E, Faes L, Wagner SK, et al. A global review of publicly available datasets for ophthalmological imaging: barriers to access, usability, and generalisability. The Lancet Digital Health. 2021 Jan 1;3(1):e51–66. Available online at: https://www.thelancet.com/journals/landig/article/PIIS2589-7500(20)30240-5/fulltext1.
Wen D, Khan SM, Xu AJ, Ibrahim H, Smith L, Caballero J, et al. Characteristics of publicly available skin cancer image datasets: a systematic review. The Lancet Digital Health. 2022 Jan 1;4(1):e64–74. Available online at: https://www.thelancet.com/journals/landig/article/PIIS2589-7500(21)00252-1/fulltext1.
Ibrahim H, Liu X, Zariffa N, Morris AD, Denniston AK. Health data poverty: an assailable barrier to equitable digital health care. The Lancet Digital Health. 2021 Apr 1;3(4):e260–5. Available online at: https://www.thelancet.com/journals/landig/article/PIIS2589-7500(20)30317-4/fulltext1.
Parikh RB, Teeple S, Navathe AS. Addressing Bias in Artificial Intelligence in Health Care. JAMA. 2019 Dec 24;322(24):2377–8. Available online at: https://jamanetwork.com/journals/jama/article-abstract/2756196 1.
Chen IY, Pierson E, Rose S, Joshi S, Ferryman K, Ghassemi M. Ethical Machine Learning in Healthcare. Annu Rev Biomed Data Sci. 2021 Jul;4:123–44. Available online at: https://www.annualreviews.org/doi/10.1146/annurev-biodatasci-092820-114757 1.
Bennett JE, Pearson-Stuttard J, Kontis V, Capewell S, Wolfe I, Ezzati M. Contributions of diseases and injuries to widening life expectancy inequalities in England from 2001 to 2016: a population-based analysis of vital registration data. The Lancet Public Health. 2018 Dec 1;3(12):e586–97. Available online at: https://www.thelancet.com/journals/lanpub/article/PIIS2468-2667(18)30214-7/fulltext 1.
Marmot M, Allen J, Boyce T, Goldblatt P, Morrison J. Health Equity in England: The Marmot Review 10 Years On - The Health Foundation [Internet]. London: Institute of Health Equity; 2020 [cited 2022 Apr 13]. Available online at: https://www.health.org.uk/publications/reports/the-marmot-review-10-years-on
Media
25 April 2022 | NHS Transformation Directorate
21 April 2022 | STANDING Together
10 November 2021 | National Cancer Research Institute
10 November 2021 | The Lancet Digital Health
22 October 2021 | University of Birmingham
20 October 2021 | The Guardian
20 October 2021 | NHSX