Abstract:In today's data-driven world, recommender systems (RS) play a crucial role to support the decision-making process. As users become continuously connected to the internet, they become less patient and less tolerant to obsolete recommendations made by an RS, e.g., movie recommendations on Netflix or books to read on Amazon. This, in turn, requires continuous training of the RS to cope with both the online fashion of data and the changing nature of user tastes and interests, known as concept drift. Streaming (online) RS has to address three requirements: continuous training and recommendation, handling concept drifts, and ability to scale. Streaming recommender systems proposed in the literature mostly, address the first two requirements and do not consider scalability. That is because they run the training process on a single machine. Such a machine, no matter how powerful it is, will eventually fail to cope with the volume of the data, a lesson learned from big data processing. To tackle the third challenge, we propose a Splitting and Replication mechanism for building distributed streaming recommender systems. Our mechanism is inspired by the successful shared-nothing architecture that underpins contemporary big data processing systems. We have applied our mechanism to two well-known approaches for online recommender systems, namely, matrix factorization and item-based collaborative filtering. We have implemented our mechanism on top of Apache Flink. We conducted experiments comparing the performance of the baseline (single machine) approach with our distributed approach. Evaluating different data sets, improvement in processing latency, throughput, and accuracy have been observed. Our experiments show online recall improvement by 40\% with more than 50\% less memory consumption.

Online Distributed Recommendation Algorithm Based on Data Stream and Peer-to-Peer Network

A Cluster-Based Incremental Recommendation Algorithm on Stream Processing Architecture

Research and Implementation of Collaborative Filtering Recommendation Algorithm Based on P2P Network

Real-time Incremental Recommendation for Streaming Data Based on Apache Flink.

A Distributed Real-Time Recommender System for Big Data Streams

A Scalable P2P Recommender System Based on Distributed Collaborative Filtering.

Real-Time Personalized Recommendation Based on Implicit User Feedback Data Stream

Online Personalized Recommendation Based On Streaming Implicit User Feedback

A hybrid recommendation algorithm based on Hadoop

Research on Parallelisation of Collaborative Filtering Recommendation Algorithm Based on Spark

A Recommendation Approach Based on Heterogeneous Network and Dynamic Knowledge Graph

A Novel Distributed Collaborative Filtering Algorithm and Its Implementation on P2P Overlay Network.

Machine Learning Based Social Media Recommendation.

Streaming Recommender Systems

Recommendation in Evolving Online Networks

Algorithm Recommendation for Data Streams

Effective Mechanism for Social Recommendation of News

Dynamic Recommendation Based on Graph Diffusion and Ebbinghaus Curve

A Distributed Collaborative Filtering Recommendation Model for P2P Networks

A Micro-Video Recommendation System Based On Big Data