Abstract:Over recent years, news recommender systems have gained significant attention in both academia and industry, emphasizing the need for a standardized benchmark to evaluate and compare the performance of these systems. Concurrently, Green AI advocates for reducing the energy consumption and environmental impact of machine learning. To address these concerns, we introduce the first Green AI benchmarking framework for news recommendation, known as GreenRec, and propose a metric for assessing the tradeoff between recommendation accuracy and efficiency. Our benchmark encompasses 30 base models and their variants, covering traditional end-to-end training paradigms as well as our proposed efficient only-encode-once (OLEO) paradigm. Through experiments consuming 2000 GPU hours, we observe that the OLEO paradigm achieves competitive accuracy compared to state-of-the-art end-to-end paradigms and delivers up to a 2992\% improvement in sustainability metrics.

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are two key challenges in news recommendation systems: 1. **Lack of standardized benchmark tests**: Currently, although news recommendation systems have received great attention in both academia and industry, there is a lack of a standardized benchmark to evaluate and compare the performance of these systems. Different studies often use their own unique methods to split and pre - process data, resulting in non - reproducible and often conflicting experimental results, making it difficult to directly compare and evaluate the results between different studies. 2. **Environmental impact and energy consumption**: As the training of large - scale neural networks has become the norm, the carbon emissions in the field of machine learning have increased significantly. Therefore, how to reduce the environmental impact of recommendation systems while ensuring their performance has become an urgent problem to be solved. To solve the above problems, the authors proposed the first news recommendation benchmark framework for green AI - GreenRec, and introduced a new metric to evaluate the trade - off between recommendation accuracy and efficiency. Through experiments, they found that the proposed Only - Encode - Once (OLEO) paradigm is not only comparable to the state - of - the - art end - to - end paradigm in terms of recommendation accuracy, but can also improve the sustainability index by up to 2992%. This indicates that the OLEO paradigm can significantly reduce the environmental impact of the recommendation system while maintaining its performance.

Benchmarking News Recommendation in the Era of Green AI