Abstract:The global infrastructure of the Web, designed as an open and transparent system, has a significant impact on our society. However, algorithmic systems of corporate entities that neglect those principles increasingly populated the Web. Typical representatives of these algorithmic systems are recommender systems that influence our society both on a scale of global politics and during mundane shopping decisions. Recently, such recommender systems have come under critique for how they may strengthen existing or even generate new kinds of biases. To this end, designers and engineers are increasingly urged to make the functioning and purpose of recommender systems more transparent. Our research relates to the discourse of algorithm awareness, that reconsiders the role of algorithm visibility in interface design. We conducted online experiments with 105 participants using MTurk for the recommender system Recoin, a gadget for Wikidata. In these experiments, we presented users with one of a set of three different designs of Recoin's user interface, each of them exhibiting a varying degree of explainability and interactivity. Our findings include a positive correlation between comprehension of and trust in an algorithmic system in our interactive redesign. However, our results are not conclusive yet, and suggest that the measures of comprehension, fairness, accuracy and trust are not yet exhaustive for the empirical study of algorithm awareness. Our qualitative insights provide a first indication for further measures. Our study participants, for example, were less concerned with the details of understanding an algorithmic calculation than with who or what is judging the result of the algorithm.

"It would be pretty immoral to choose a random algorithm"

People Prefer Moral Discretion to Algorithms: Algorithm Aversion Beyond Intransparency

Transparency as design publicity: explaining and justifying inscrutable algorithms

Transparency in Algorithmic and Human Decision-Making: Is There a Double Standard?

Explaining Decision-Making Algorithms through UI: Strategies to Help Non-Expert Stakeholders

Are algorithms always fair? The study on public preferences toward algorithmic decision‐making: A case study from the perspectives of decision scenarios and social roles

Perceived algorithmic fairness: An empirical study of transparency and anthropomorphism in algorithmic recruiting

Unboxing the Black Box of Artificial Intelligence: Algorithmic Transparency and/or a Right to Functional Explainability

Perception of fairness in algorithmic decisions: Future developers' perspective

The ethics of algorithms: Mapping the debate

Fairness perceptions of algorithmic decision-making: A systematic review of the empirical literature

Human Perceptions of Fairness in Algorithmic Decision Making: A Case Study of Criminal Risk Prediction

Will Algorithms Blind People? The Effect of Explainable AI and Decision-Makers’ Experience on AI-supported Decision-Making in Government

Discrimination in the Age of Algorithms

Examining the Impact of Algorithm Awareness on Wikidata's Recommender System Recoin

Influence of the algorithm's reliability and transparency in the user's decision-making process

The shadow of the algorithm: the ethical blind spot of artificial intelligence education

Automated news recommendation in front of adversarial examples and the technical limits of transparency in algorithmic accountability

What Are You Hiding? Algorithmic Transparency and User Perceptions

Dimensions of Diversity in Human Perceptions of Algorithmic Fairness

Keeping Designers in the Loop: Communicating Inherent Algorithmic Trade-offs Across Multiple Objectives