M3oE: Multi-Domain Multi-Task Mixture-of Experts Recommendation Framework

Zijian Zhang,Shuchang Liu,Jiaao Yu,Qingpeng Cai,Xiangyu Zhao,Chunxu Zhang,Ziru Liu,Qidong Liu,Hongwei Zhao,Lantao Hu,Peng Jiang,Kun Gai

DOI: https://doi.org/10.1145/3626772.3657686

2024-05-12

Abstract:Multi-domain recommendation and multi-task recommendation have demonstrated their effectiveness in leveraging common information from different domains and objectives for comprehensive user modeling. Nonetheless, the practical recommendation usually faces multiple domains and tasks simultaneously, which cannot be well-addressed by current methods. To this end, we introduce M3oE, an adaptive Multi-domain Multi-task Mixture-of-Experts recommendation framework. M3oE integrates multi-domain information, maps knowledge across domains and tasks, and optimizes multiple objectives. We leverage three mixture-of-experts modules to learn common, domain-aspect, and task-aspect user preferences respectively to address the complex dependencies among multiple domains and tasks in a disentangled manner. Additionally, we design a two-level fusion mechanism for precise control over feature extraction and fusion across diverse domains and tasks. The framework's adaptability is further enhanced by applying AutoML technique, which allows dynamic structure optimization. To the best of the authors' knowledge, our M3oE is the first effort to solve multi-domain multi-task recommendation self-adaptively. Extensive experiments on two benchmark datasets against diverse baselines demonstrate M3oE's superior performance. The implementation code is available to ensure reproducibility.

Information Retrieval,Artificial Intelligence

What problem does this paper attempt to address?

The paper attempts to address the challenging problem in multi-domain multi-task recommendation systems. Specifically, existing multi-domain (MDR) or multi-task (MTR) recommendation methods often only partially solve the problem in practical applications and perform poorly in certain specific situations. The authors observe that when dealing with multiple domains and multiple tasks simultaneously, existing MDR and MTR methods struggle to achieve optimal or balanced performance. This phenomenon is referred to as the "multi-domain multi-task seesaw" (MDMT seesaw), which manifests in:

1. **The same multi-domain information transfer method may not generalize to different tasks**: For example, on a video sharing platform, when transferring user preferences from the TV domain to the tablet domain, different tasks (such as watching, liking) may be affected to varying degrees.
2. **The same multi-task optimization strategy may not generalize to different domains**: Similarly, on the same platform, the balancing strategy between user behaviors (such as watching, liking) in different domains may not be applicable to other domains.

To address this issue, the authors propose a new framework called M3oE, which can handle multi-domain information extraction and multi-task inference simultaneously, thereby achieving better knowledge transfer and integration. By introducing three expert modules (shared expert module S, domain expert module D, task expert module T), M3oE can capture information from different perspectives in a decoupled manner and use a two-level fusion mechanism to control the aggregation of information for each domain and task. Additionally, by using AutoML technology to dynamically optimize fusion weights, the model can achieve higher adaptability in different datasets and recommendation environments.

Experimental results show that M3oE outperforms the current state-of-the-art methods on two benchmark datasets, and in-depth analysis supports its effectiveness in knowledge decoupling and integration.