Towards a Conversational Measure of Trust

Mengyao Li,Areen Alsaid,Sofia I. Noejovich,Ernest V. Cross,John D. Lee

DOI: https://doi.org/10.48550/arXiv.2010.04885

2020-10-10

Abstract:The increasingly collaborative decision-making process between humans and agents demands a comprehensive, continuous, and unobtrusive measure of trust in agents. The gold standard format for measuring trust, a Likert-style survey, suffers from major limitations in dynamic human-agent interactions. We proposed a new approach to evaluate trust in a nondirective and relational conversation. The term nondirective refers to abstract word selections in open-ended prompts, which can probe respondents to freely describe their attitudes. The term relational refers to interactive conversations where respondents can clarify their responses in followup questions. We propose a systematic process for generating nondirective trust-based prompts by using text analysis from previously validated trust scales. This nondirective and relational approach provides a complementary trust measurement, which can unobtrusively elicit rich and dynamic information on situational trust throughout a human-agent interaction.

Human-Computer Interaction

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the limitations of existing trust measurement methods in dynamic human - machine interaction. Specifically, traditional trust measurement methods, such as the Likert - scale, have the following main problems when evaluating the trust between humans and automated systems:

1. **Interruptiveness**: Traditional trust questionnaires usually appear in text form and require participants to interrupt their activities during the interaction with the machine to complete the questionnaire. This will interfere with the natural interaction process and it is difficult to capture the dynamic characteristics of trust changing over time.

2. **Leading - bias**: Direct descriptive statements (such as "the system is suspicious") may lead to the anchoring effect, that is, participants rely on these preset information to judge their trust in the system instead of basing on their own actual experiences and feelings.

3. **Positive - bias**: Many commonly used scales are unbalanced in design and tend to produce positive evaluations, which may mask the real attitudes of users.

4. **Obsolescence**: With the development of technology, the nature of the human - machine relationship has changed, and some past trust scales may no longer be applicable to the current human - machine interaction situations.

To solve these problems, the paper proposes a new trust measurement method - non - directive and relational conversational trust measurement. This method evaluates trust through open - ended questions and interactive conversations, allowing participants to freely express their views and feelings about the automated system, thus providing more abundant and dynamic trust information. In addition, this conversational measurement method can be carried out without disturbing users from performing their main tasks, for example, evaluating trust during driving, making trust measurement more natural and non - intrusive.