Abstract:To safely deploy deep learning-based computer vision models for computer-aided detection and diagnosis, we must ensure that they are robust and reliable. Towards that goal, algorithmic auditing has received substantial attention. To guide their audit procedures, existing methods rely on heuristic approaches or high-level objectives (e.g., non-discrimination in regards to protected attributes, such as sex, gender, or race). However, algorithms may show bias with respect to various attributes beyond the more obvious ones, and integrity issues related to these more subtle attributes can have serious consequences. To enable the generation of actionable, data-driven hypotheses which identify specific dataset attributes likely to induce model bias, we contribute a first technique for the rigorous, quantitative screening of medical image datasets. Drawing from literature in the causal inference and information theory domains, our procedure decomposes the risks associated with dataset attributes in terms of their detectability and utility (defined as the amount of information knowing the attribute gives about a task label). To demonstrate the effectiveness and sensitivity of our method, we develop a variety of datasets with synthetically inserted artifacts with different degrees of association to the target label that allow evaluation of inherited model biases via comparison of performance against true counterfactual examples. Using these datasets and results from hundreds of trained models, we show our screening method reliably identifies nearly imperceptible bias-inducing artifacts. Lastly, we apply our method to the natural attributes of a popular skin-lesion dataset and demonstrate its success. Our approach provides a means to perform more systematic algorithmic audits and guide future data collection efforts in pursuit of safer and more reliable models.

Data Provenance via Differential Auditing

Cognitive Tracing Data Trails: Auditing Data Provenance in Discriminative Language Models Using Accumulated Discrepancy Score

A General Framework for Data-Use Auditing of ML Models

A General Framework for Auditing Differentially Private Machine Learning

Tight Auditing of Differentially Private Machine Learning

TeDA: A Testing Framework for Data Usage Auditing in Deep Learning Model Development

Debugging Differential Privacy: A Case Study for Privacy Auditing

Curator Attack: When Blackbox Differential Privacy Auditing Loses Its Power

Nearly Tight Black-Box Auditing of Differentially Private Machine Learning

Insights into Data through Model Behaviour: An Explainability-driven Strategy for Data Auditing for Responsible Computer Vision Applications

Hide in Plain Sight: Clean-Label Backdoor for Auditing Membership Inference

SoK: Dataset Copyright Auditing in Machine Learning Systems

XAudit : A Theoretical Look at Auditing with Explanations

Provenance of Training Without Training Data: Towards Privacy-Preserving DNN Model Ownership Verification

EMA: Auditing Data Removal from Trained Models

Auditing Differential Privacy Guarantees Using Density Estimation

Auditing Differentially Private Machine Learning: How Private is Private SGD?

Model Provenance via Model DNA

Auditing Private Prediction

Auditing $f$-Differential Privacy in One Run

Data AUDIT: Identifying Attribute Utility- and Detectability-Induced Bias in Task Models