Abstract:Reciprocal recommender systems~(RRS), conducting bilateral recommendations between two involved parties, have gained increasing attention for enhancing matching efficiency. However, the majority of existing methods in the literature still reuse conventional ranking metrics to separately assess the performance on each side of the recommendation process. These methods overlook the fact that the ranking outcomes of both sides collectively influence the effectiveness of the RRS, neglecting the necessity of a more holistic evaluation and a capable systemic solution. In this paper, we systemically revisit the task of reciprocal recommendation, by introducing the new metrics, formulation, and method. Firstly, we propose five new evaluation metrics that comprehensively and accurately assess the performance of RRS from three distinct perspectives: overall coverage, bilateral stability, and balanced ranking. These metrics provide a more holistic understanding of the system's effectiveness and enable a comprehensive evaluation. Furthermore, we formulate the RRS from a causal perspective, formulating recommendations as bilateral interventions, which can better model the decoupled effects of potential influencing factors. By utilizing the potential outcome framework, we further develop a model-agnostic causal reciprocal recommendation method that considers the causal effects of recommendations. Additionally, we introduce a reranking strategy to maximize matching outcomes, as measured by the proposed metrics. Extensive experiments on two real-world datasets from recruitment and dating scenarios demonstrate the effectiveness of our proposed metrics and approach. The code and dataset are available at: <a class="link-external link-https" href="https://github.com/RUCAIBox/CRRS" rel="external noopener nofollow">this https URL</a>.

Overview of the Evaluated Algorithms for the Personal Recommendation Systems

Evaluation Metrics for Personalized Recommendation Systems

RECENT ADVANCES IN PERSONAL RECOMMENDER SYSTEMS

How good your recommender system is? A survey on evaluations in recommendation

User Interest Dynamics on Personalized Recommendation

A Comprehensive Survey of Evaluation Techniques for Recommendation Systems

A Personalized Recommendation Algorithm Via Biased Random Walk

An Improved Network-Based Recommendation Model Via Inhibiting Algorithm Bias

Goal Modeling-based Evaluation for Personalized Recommendation Systems

Adaptive recommendation systems: A comparative analysis of KNN-based algorithms and hybrid models

Toward a New Protocol to Evaluate Recommender Systems

A Joint Optimization Approach for Personalized Recommendation Diversification.

Research and implementation of a personalized recommendation system

Revisiting Reciprocal Recommender Systems: Metrics, Formulation, and Method

A Revisiting Study of Appropriate Offline Evaluation for Top- N Recommendation Algorithms

Towards Unified Metrics for Accuracy and Diversity for Recommender Systems

Top-N Recommendation Algorithms: A Quest for the State-of-the-Art

A Personalized Recommendation Approach Based on Content Similarity Calculation in Large-Scale Data

A Systematical Evaluation for Next-Basket Recommendation Algorithms

A New Multi-criteria Recommendation Algorithm for Groups of Users.

Measuring "Why" in Recommender Systems: a Comprehensive Survey on the Evaluation of Explainable Recommendation