Abstract:The opaque nature of transformer-based models, particularly in applications susceptible to unethical practices such as dark-patterns in user interfaces, requires models that integrate uncertainty quantification to enhance trust in predictions. This study focuses on dark-pattern detection, deceptive design choices that manipulate user decisions, undermining autonomy and consent. We propose a differential fine-tuning approach implemented at the final classification head via uncertainty quantification with transformer-based pre-trained models. Employing a dense neural network (DNN) head architecture as a baseline, we examine two methods capable of quantifying uncertainty: Spectral-normalized Neural Gaussian Processes (SNGPs) and Bayesian Neural Networks (BNNs). These methods are evaluated on a set of open-source foundational models across multiple dimensions: model performance, variance in certainty of predictions and environmental impact during training and inference phases. Results demonstrate that integrating uncertainty quantification maintains performance while providing insights into challenging instances within the models. Moreover, the study reveals that the environmental impact does not uniformly increase with the incorporation of uncertainty quantification techniques. The study's findings demonstrate that uncertainty quantification enhances transparency and provides measurable confidence in predictions, improving the explainability and clarity of black-box models. This facilitates informed decision-making and mitigates the influence of dark-patterns on user interfaces. These results highlight the importance of incorporating uncertainty quantification techniques in developing machine learning models, particularly in domains where interpretability and trustworthiness are critical.

Uncertainty Estimation of Transformers' Predictions via Topological Analysis of the Attention Matrices

Revisiting Uncertainty-based Query Strategies for Active Learning with Transformers

Uncertainty Quantification for Transformer Models for Dark-Pattern Detection

Towards trustworthy rotating machinery fault diagnosis via attention uncertainty in transformer

Transformer-Based Astronomical Time Series Model with Uncertainty Estimation for Detecting Misclassified Instances

Uncertainty-Aware Machine Translation Evaluation

Representational Strengths and Limitations of Transformers

Looking Beyond The Top-1: Transformers Determine Top Tokens In Order

Uncertainty-Guided Transformer Reasoning for Camouflaged Object Detection.

Comparing and Improving Active Learning Uncertainty Measures for Transformer Models by Discarding Outliers

AttentionViz: A Global View of Transformer Attention

Just rotate it! Uncertainty estimation in closed-source models via multiple queries

On Uncertainty In Natural Language Processing

Calibrating Transformers via Sparse Gaussian Processes

Uncertainty-guided transformer for brain tumor segmentation

Unveiling and Controlling Anomalous Attention Distribution in Transformers

Towards Better Understanding of In-Context Learning Ability from In-Context Uncertainty Quantification

Pixel-Level Anomaly Detection Via Uncertainty-aware Prototypical Transformer

Generalized Probabilistic Attention Mechanism in Transformers

Unmasking Transformers: A Theoretical Approach to Data Recovery via Attention Weights