Multi-Model Assessing and Visualizing Consistency and Compatibility of Experts in Group Decision-Making

Bojan Srđević,Zorica Srđević
DOI: https://doi.org/10.3390/math12111699
IF: 2.4
2024-05-31
Mathematics
Abstract:In this paper, an approach is proposed for assessing the performance of experts in the group from two perspectives: (1) individual consistencies and (2) deviations from the group decision. The quality of performance of the experts is based on combining the standard and rough analytic hierarchy process (AHP) with the technique for order of preference by similarity to the ideal solution (TOPSIS). The statistical method CRITIC is used to derive weights for the TOPSIS method before the experts are assessed based on demonstrated consistency and deviations from the group. Common performance indicators, such as consistency ratio, Euclidean distance, compatibility, and Spearman's correlation coefficient, are proposed for re-grouping experts before making the final decisions. A genetic algorithm enables the efficient solving of this complex clustering problem. Implementing the described approach and method can be useful in comparable assessment frameworks. A critical aspect is conducting a thorough pre-assessment of the competence of potential decision makers, often referred to as experts who may not consistently exhibit apparent expertise. The competence of decision makers (which does not have to be associated with compatibility) is evidenced by selected consistency parameters, and in a way, a pre-assessment of their competence follows Plato's 'government of the wise' principle. In the presented study, the compatibility of individuals in the group with the collective position (group decision) is measured by parameters related to their compatibility with the group solution and statistical deviation while ranking decision elements. The proposed multi-model-based approach stands out for its resilience in conducting thorough pre-assessment of the quality (competence) of potential decision makers, often regarded as experts who might not consistently display evident expertise. The wetland study area in Serbia is used as an example application, where seven measures for reducing the risk of drought were evaluated by twelve experts coming from different sectors and with different backgrounds and expertise.
mathematics
What problem does this paper attempt to address?
The problem that this paper attempts to solve is to evaluate the quality of expert performance in the process of group decision - making. Specifically, the paper focuses on two main aspects: 1. **Individual Consistency**: Evaluate the individual consistency of each expert in the decision - making process. This involves checking the stability and reliability of the expert in their judgments. 2. **Deviation from Group Decision**: Evaluate the deviation between each expert's decision and the final group decision. This helps to identify those experts who are significantly different from the group decision. To achieve these goals, the paper proposes a method that combines the Analytic Hierarchy Process (AHP) and the Rough Analytic Hierarchy Process (Rough AHP), and uses the Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) method for comprehensive evaluation. In addition, the CRITIC method is also used to determine the weights of performance indicators, and the genetic algorithm is used to efficiently solve complex clustering problems. ### Specific Problem Description 1. **Selection of the Number of Experts and Evaluation Tools**: - Determine the number of experts participating in group decision - making. - Select appropriate tools to evaluate the consistency and deviation of experts. 2. **Measurement of Deviation between Individual and Group Decisions**: - Once the group decision is obtained through a certain aggregation method, how to measure the deviation between individual and group decisions. - These deviation measurements can help determine the optimal size of the expert group, making the decision - making process more efficient. 3. **Measurement of Consistency and Compatibility**: - Use specific indicators (such as consistency ratio, Euclidean distance, compatibility, and Spearman correlation coefficient) to evaluate the performance of experts. - These indicators are helpful for regrouping experts before making the final decision. 4. **Handling Challenges in Group Decision - making**: - Cope with the challenges brought by the diverse backgrounds, different attitudes, and different communication and adaptability of group members. - Decision - makers need to continuously adjust their judgments throughout the decision - making process based on new information. ### Method Overview - **Standard AHP and Rough AHP**: Used to obtain individual priority vectors and generate group vectors through geometric aggregation. - **TOPSIS Method**: Used to evaluate and rank the quality of experts' decision - making performance. - **CRITIC Method**: Used to determine the objective weights of performance indicators. - **Genetic Algorithm**: Used to map experts to different clusters according to performance indicators. ### Application Example The paper takes the evaluation of measures to reduce drought risk in wetland management along the Danube in Serbia as an example to show how to apply the proposed multi - model method. 12 experts from different departments, with different educational backgrounds and professional skills, participated in the evaluation process. Through this method, the consistency of experts and the deviation from group decision can be effectively identified, thereby optimizing the group decision - making process.