Abstract:Trust is a crucial component in collaborative multiagent systems (MAS) involving humans and autonomous AI agents. Rather than assuming trust based on past system behaviours, it is important to formally verify trust by modelling the current state and capabilities of agents. We argue for verifying actual trust relations based on agents abilities to deliver intended outcomes in specific contexts. To enable reasoning about different notions of trust, we propose using the refinement-based formal method Event-B. Refinement allows progressively introducing new aspects of trust from abstract to concrete models incorporating knowledge and runtime states. We demonstrate modelling three trust concepts and verifying associated trust properties in MAS. The formal, correctness-by-construction approach allows to deduce guarantees about trustworthy autonomy in human-AI partnerships. Overall, our contribution facilitates rigorous verification of trust in multiagent systems.
What problem does this paper attempt to address?
The problem that this paper attempts to solve is: in a collaborative multi - agent system (MAS) involving humans and autonomous AI agents, how to ensure the credibility of the system through formal verification of trust relationships. Specifically, the paper focuses on how to formally verify trust based on the current state and capabilities of agents, rather than relying solely on past behavior.
### Core of the Problem
1. **Limitations of Traditional Trust Modeling**
- Traditional trust modeling methods mainly rely on reasoning about past behavior, and this method may not accurately reflect the actual capabilities and intentions of agents in the current situation.
- In a dynamic environment, relying solely on historical data is not sufficient to assess whether an agent can complete a specific task.
2. **Requirement for Actual Trust**
- The paper proposes a concept called "actual trust", that is, to evaluate whether an agent can complete a specific task according to its current capabilities, knowledge, and commitment.
- Actual trust emphasizes whether an agent can truly deliver the expected results in a specific situation, rather than trust based solely on historical performance.
3. **Necessity of Verifying Trust**
- In human - machine collaboration, especially in safety - critical applications, formal verification of trust is crucial. For example, an autonomous delivery vehicle (ADV) not only needs to be verified offline during the design stage to ensure basic safety, but also needs to be evaluated online whether its current state (such as battery power, load, etc.) can meet the requirements of specific tasks.
### Solution
To achieve this goal, the paper introduces a formal method based on Event - B, especially through the step - by - step refinement technique to model and verify trust:
- **Abstract Model**: First, an abstract trust relationship model is defined, which describes the basic trust relationship between the trustor and the trustee.
- **Step - by - Step Refinement**:
- **First - level Refinement**: The knowledge of the trustor is introduced, considering the trustor's perception of the trustee's capabilities.
- **Second - level Refinement**: Further introduce the trustee's commitment to ensure that the trustee is not only capable but also willing to perform specific tasks.
In this way, the paper shows how to formally model and verify three trust concepts in a multi - agent system: strategic trust, cognitive trust, and commitment trust.
### Formulas and Symbol Representations
The paper uses the formal language of Event - B, which includes set theory and first - order logic. Here are the Markdown - format representations of several key formulas:
- **Definition of Trustor and Trustee**
\[
\text{CONTEXT cntx0} \\
\text{SETS AGENTS TASKS} \\
\text{CONSTANTS trustors trustees} \\
\text{AXIOMS axm1 : } \text{trustors} \subseteq \text{AGENTS} \\
\text{axm2 : } \text{trustees} \subseteq \text{AGENTS} \\
\text{axm3 : partition(AGENTS, trustors, trustees)}
\]
- **Abstract Trust Relationship**
\[
\text{Definition 1. Abstract (strategic) trust: agent } i \text{ weakly (abstraction) trusts } j \text{ regarding task } t \text{ if } j \text{ has an action or a sequence of actions available to it to ensure } e.
\]
\[
\text{Trust is modelled as a three - dimension relation variable between a trustor, a set of trustees and }
\]