A value-sensitive design approach to minimize value tensions in software-based risk-assessment instruments

Nick Lüthi,Christian Matt,Thomas Myrach
DOI: https://doi.org/10.1080/12460125.2020.1859744
2021-02-28
Journal of Decision Systems
Abstract:<span>Many authorities currently use software-based risk-assessment instruments (SBRAIs) to assess a person's potential to commit crimes (e.g. terrorism, sexual assault). SBRAIs promise better, faster, and more consistent decision-making than their paper-based predecessors. The underlying SBRAI classification schema is well-understood, but given the potentially serious ethical consequences of misjudgments, there is a lack of understanding of how a newly introduced IT artefact, or digital transformation, affects the overall process. Applying a value-centric lens and using Value Sensitive Design (VSD), we identify users' most relevant SBRAI values, as well as the existing value tensions between developers and users. We find that various value tensions, which had not been an issue prior to digitization, occur. These are tensions regarding the values traceability, reliability, neutrality, support, and trust. These value tensions often emerge due to misunderstandings and miscommunications that unclear classifications enable. Practitioners should therefore focus on four design guidelines to avoid SBRAI value tensions: constant and accessible information, transparency, isolated use anticipation, and compliance with novice and expert users' requirements.</span>
What problem does this paper attempt to address?