Abstract:Machine learning technologies are increasingly developed for use in healthcare. While research communities have focused on creating state-of-the-art models, there has been less focus on real world implementation and the associated challenges to accuracy, fairness, accountability, and transparency that come from actual, situated use. Serious questions remain under examined regarding how to ethically build models, interpret and explain model output, recognize and account for biases, and minimize disruptions to professional expertise and work cultures. We address this gap in the literature and provide a detailed case study covering the development, implementation, and evaluation of Sepsis Watch, a machine learning-driven tool that assists hospital clinicians in the early diagnosis and treatment of sepsis. We, the team that developed and evaluated the tool, discuss our conceptualization of the tool not as a model deployed in the world but instead as a socio-technical system requiring integration into existing social and professional contexts. Rather than focusing on model interpretability to ensure a fair and accountable machine learning, we point toward four key values and practices that should be considered when developing machine learning to support clinical decision-making: rigorously define the problem in context, build relationships with stakeholders, respect professional discretion, and create ongoing feedback loops with stakeholders. Our work has significant implications for future research regarding mechanisms of institutional accountability and considerations for designing machine learning systems. Our work underscores the limits of model interpretability as a solution to ensure transparency, accuracy, and accountability in practice. Instead, our work demonstrates other means and goals to achieve FATML values in design and in practice.

Embodied Machine Learning

From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence

A First Step in Using Machine Learning Methods to Enhance Interaction Analysis for Embodied Learning Environments

The promise and peril of interactive embodied agents for studying non-verbal communication: a machine learning perspective

Embodied Artificial Intelligence: Enabling the Next Intelligence Revolution

Trustworthy Machine Learning under Social and Adversarial Data Sources

Monitoring Trust in Human-Machine Interactions for Public Sector Applications

Dancing With Algorithms: Interaction Creates Greater Preference and Trust in Machine-Learned Behavior

Confidence Estimation Using Machine Learning in Immersive Learning Environments

ICE: Enabling Non-Experts to Build Models Interactively for Large-Scale Lopsided Problems

Designs for Enabling Collaboration in Human-Machine Teaming via Interactive and Explainable Systems

Reinforcement Learning based Embodied Agents Modelling Human Users Through Interaction and Multi-Sensory Perception

Computational Modeling of the Dynamics of Human Trust During Human–Machine Interactions

Machine Learning practices and infrastructures

Enhancing Human-Machine Teaming for Medical Prognosis Through Neural Ordinary Differential Equations (NODEs)

A Diachronic Perspective on User Trust in AI under Uncertainty

Trust as Extended Control: Active Inference and User Feedback During Human-Robot Collaboration

Incorporating Experts' Judgment into Machine Learning Models

"The Human Body is a Black Box": Supporting Clinical Decision-Making with Deep Learning

SoK: Machine Learning Governance

Embodied Artificial Intelligence through Distributed Adaptive Control: An Integrated Framework