User-Centric Design and Evaluation of Online Interactive Recommender Systems

Qian Zhao

2018-01-01

Abstract:University of Minnesota Ph.D. dissertation.May 2018. Major: Computer Science. Advisor: Joseph Konstan. 1 computer file (PDF); xii, 145 pages.

What problem does this paper attempt to address?

A Novel Framework To Process The Quantity And Quality Of User Behavior Data In Recommender Systems

Penghua Yu,Lanfen Lin,Yuangang Yao

DOI: https://doi.org/10.1007/978-3-319-39937-9_18

2016-01-01

Abstract:Recommender system has become one of the most popular techniques to cope with the information overload problem. In the past years many algorithms have been proposed to obtain accurate recommendations. Such methods usually put all the collected user data into learning models without a careful consideration of the quantity and quality of individual user feedbacks. Yet in real applications, different types of users tend to represent preferences and opinions in various ways, thus resulting in user data with radically diverse quantity and quality. This characteristic of data influences the performance of recommendations. However, little attention has been devoted to the management of quantity and quality for user data in recommender systems. In this paper, we propose a generic framework to seamlessly exploit different pre-processing and recommendation approaches for ratings of different users. More specifically, we first classify users into groups based on the quantity and quality of their behavior data. In order to handle the user groups diversely, we further propose several data pre-processing strategies. Subsequently, we present a novel transfer latent factor model (TLMF) to transfer learnt models between groups. Finally, we conduct extensive experiments on a large data set and demonstrate the effectiveness of our proposed framework.
Optimizing the Design of Recommendation Systems.

Xiaohan Guo,Dejun Xie

DOI: https://doi.org/10.1145/3479162.3479174

2021-01-01

Abstract:With the development of digital technology and internet applications, the massive market data and information overload make it hard for users to organize and make good use of comments to choose their preferred items. In this connection, recommendation systems are widely developed with high expectations. While there have been substantial researches focusing on the recommendation methods and implementations, this paper provides an up-to-date synthesis of the essential factors towards a more rigorous, user-helping system design in consolidation of collaborative filtering, content-based model, and hybrid model in a fast-developing web and social network environment. Aspects of evaluation metrics are analyzed and compared in connection to particular applicabilities in the system design. Further explorations in terms of credit rating are discussed in a real-business setting.
RECENT ADVANCES IN PERSONAL RECOMMENDER SYSTEMS

JIAN-GUO LIU,MICHAEL Z. Q. CHEN,JIANCHI CHEN,FEI DENG,HAI-TAO ZHANG,ZI-KE ZHANG,TAO ZHOU

2009-01-01

Abstract:In the past years we have witnessed an explosive growth of the data and information on the World Wide Web, which makes it difficult for normal users to find the information that they are interested in. On the other hand, the majority of the data and resources are very unpopular, which can be considered as "hidden information", and are very difficult tofind. By building a bridge between the users and the objects and constructing their similarities, the Personal Recommender System (PRS) can recommend the objects that the users are potentially interested in. PRS plays an important role in not only social and economic life but also scientific analysis. The interdisciplinary PRS attracts attention from the communities of information science, computational mathematics, statistical physics, management science, and consumer behaviors, etc. In fact, PRS is one of the most efficient tools to solve the i nformation over- load problem. According to the recommendation algorithms, we introduce four typical systems, including the collaborating filtering system, the content-based system, the structure-based system, and the hybrid system. In addition, some improved algorithms are proposed to overcome the limitations of traditional systems. This review article may shed some light on the study of PRS from different backgrounds.
Personal Recommendation in User-Object Networks

Tao Zhou

DOI: https://doi.org/10.1007/978-3-642-02466-5_23

2009-01-01

Abstract:Thanks to the Internet and the World Wide Web, we live in a world of many possibilities we can choose from thousands of movies, millions of books, and billions of web pages. Far exceeding our personal processing capacity, this excessive freedom of choice calls for automated ways to find the relevant information. As a result, the field of information filtering is very active and rich with unanswered challenges. In this short paper, I will give a brief introduction on the design of recommender systems, which recommend objects to users based on the historical records of users' activities. A diffusion-based recommendation algorithm, as well as two improved algorithms are investigated. Numerical results on a benchmark data set have demonstrated the advantages in algorithmic accuracy.
User Interest Dynamics on Personalized Recommendation

Tian Qiu,Chi Wan,Xiao-Fan Wang,Zi-Ke Zhang

DOI: https://doi.org/10.1016/j.physa.2019.03.113

2019-01-01

Abstract:Four real recommender system datasets, the Netflix, SMovieLens, LMovieLens and RYM datasets, are analyzed to gain an insight into their user interest characteristics. A preference of active users to cold objects and a diverse interest of inactive users are revealed, which characteristics are introduced to improve the personalized recommendation algorithms. Based on seven different algorithms, we propose a general improvement formula for them, and finally four new algorithms are obtained. Tested on the four datasets, all the new algorithms are found to outperform the original ones in recommendation accuracy, diversity and novelty, except for the diversity and novelty compared with a heat conduction algorithm. And the recommendation accuracy for the cold objects, referring to the objects with small degrees, is also improved. Moreover, one of the new algorithms shows better performance than two other excellent algorithms in many aspects, i.e., the hybrid algorithm of heat conduction and mass diffusion, and the biased heat conduction algorithm. Our work may shed a new light on personalized recommendation from the perspective of connecting empirical study with algorithm design.
Evaluating Conversational Recommender Systems: A Landscape of Research

Dietmar Jannach

DOI: https://doi.org/10.1007/s10462-022-10229-x

2022-10-28

Abstract:Conversational recommender systems aim to interactively support online users in their information search and decision-making processes in an intuitive way. With the latest advances in voice-controlled devices, natural language processing, and AI in general, such systems received increased attention in recent years. Technically, conversational recommenders are usually complex multi-component applications and often consist of multiple machine learning models and a natural language user interface. Evaluating such a complex system in a holistic way can therefore be challenging, as it requires (i) the assessment of the quality of the different learning components, and (ii) the quality perception of the system as a whole by users. Thus, a mixed methods approach is often required, which may combine objective (computational) and subjective (perception-oriented) evaluation techniques. In this paper, we review common evaluation approaches for conversational recommender systems, identify possible limitations, and outline future directions towards more holistic evaluation practices.

Information Retrieval,Human-Computer Interaction
Uncovering the Information Core in Recommender Systems

Wei Zeng,An Zeng,Hao Liu,Ming-Sheng Shang,Tao Zhou

DOI: https://doi.org/10.1038/srep06140

IF: 4.6

2014-01-01

Scientific Reports

Abstract:With the rapid growth of the Internet and overwhelming amount of information that people are confronted with, recommender systems have been developed to effectively support users' decision-making process in online systems. So far, much attention has been paid to designing new recommendation algorithms and improving existent ones. However, few works considered the different contributions from different users to the performance of a recommender system. Such studies can help us improve the recommendation efficiency by excluding irrelevant users. In this paper, we argue that in each online system there exists a group of core users who carry most of the information for recommendation. With them, the recommender systems can already generate satisfactory recommendation. Our core user extraction method enables the recommender systems to achieve 90% of the accuracy of the top- L recommendation by taking only 20% of the users into account. A detailed investigation reveals that these core users are not necessarily the large-degree users. Moreover, they tend to select high quality objects and their selections are well diversified.
Human-Centered Recommender Systems: Origins, Advances, Challenges, and Opportunities

Joseph Konstan,Loren Terveen

DOI: https://doi.org/10.1609/aimag.v42i3.18142

IF: 0.9

2021-11-20

AI Magazine

Abstract:From the earliest days of the field, Recommender Systems research and practice has struggled to balance and integrate approaches that focus on recommendation as a machine learning or missing-value problem with ones that focus on machine learning as a discovery tool and perhaps persuasion platform. In this article, we review 25 years of recommender systems research from a human-centered perspective, looking at the interface and algorithm studies that advanced our understanding of how system designs can be tailored to users objectives and needs. At the same time, we show how external factors, including commercialization and technology developments, have shaped research on human-centered recommender systems. We show how several unifying frameworks have helped developers and researchers alike incorporate thinking about user experience and human decision-making into their designs. We then review the challenges, and the opportunities, in today’s recommenders, looking at how deep learning and optimization techniques can integrate with both interface designs and human performance statistics to improve recommender effectiveness and usefulness

computer science, artificial intelligence
How good your recommender system is? A survey on evaluations in recommendation

Thiago Silveira,Min Zhang,Xiao Lin,Yiqun Liu,Shaoping Ma

DOI: https://doi.org/10.1007/s13042-017-0762-9

2017-12-14

International Journal of Machine Learning and Cybernetics

Abstract:Recommender Systems have become a very useful tool for a large variety of domains. Researchers have been attempting to improve their algorithms in order to issue better predictions to the users. However, one of the current challenges in the area refers to how to properly evaluate the predictions generated by a recommender system. In the extent of offline evaluations, some traditional concepts of evaluation have been explored, such as accuracy, Root Mean Square Error and P@N for top-k recommendations. In recent years, more research have proposed some new concepts such as novelty, diversity and serendipity. These concepts have been addressed with the goal to satisfy the users’ requirements. Numerous definitions and metrics have been proposed in previous work. On the absence of a specific summarization on evaluations of recommendation combining traditional metrics and recent progresses, this paper surveys and organizes the main research that present definitions about concepts and propose metrics or strategies to evaluate recommendations. In addition, this survey also settles the relationship between the concepts, categorizes them according to their objectives and suggests potential future topics on user satisfaction.

computer science, artificial intelligence
Analysis and Design of a Personalized Recommendation System Based on a Dynamic User Interest Model

Chunyan Mao,Shuaishuai Huang,Mingxiu Sui,Haowei Yang,Xueshe Wang

2024-10-14

Abstract:With the rapid development of the internet and the explosion of information, providing users with accurate personalized recommendations has become an important research topic. This paper designs and analyzes a personalized recommendation system based on a dynamic user interest model. The system captures user behavior data, constructs a dynamic user interest model, and combines multiple recommendation algorithms to provide personalized content to users. The research results show that this system significantly improves recommendation accuracy and user satisfaction. This paper discusses the system's architecture design, algorithm implementation, and experimental results in detail and explores future research directions.

Information Retrieval,Artificial Intelligence
Product and User Dependent Social Network Models for Recommender Systems.

Min Li,Zhiwei Jiang,Bin Luo,Jiubin Tang,Qing Gu,Daoxu Chen

DOI: https://doi.org/10.1007/978-3-642-37456-2_2

2013-01-01

Abstract:Social network based applications such as Facebook, Myspace and LinkedIn have become very popular among Internet users, and a major research problem is how to use the social network information to better infer users' preferences and make better recommender systems. A common trend is combining the user-item rating matrix and users' social network for recommendations. However, existing solutions add the social network information for a particular user without considering the different characteristics of the products to be recommended and the neighbors involved. This paper proposes a new approach that can adaptively utilize social network information based on the context characterized by products and users. This approach complements several existing social network based recommendation algorithms and thus can be integrated with existing solutions. Experimental results on Epinions data set demonstrate the added value of the proposed solution in two recommendation tasks: rating prediction and top-K recommendations. © Springer-Verlag 2013.
Exploring Mental Models for Transparent and Controllable Recommender Systems: A Qualitative Study

Thao Ngo,Johannes Kunkel,Jürgen Ziegler

DOI: https://doi.org/10.1145/3340631.3394841

2020-07-07

Abstract:While online content is personalized to an increasing degree, eg. using recommender systems (RS), the rationale behind personalization and how users can adjust it typically remains opaque. This was often observed to have negative effects on the user experience and perceived quality of RS. As a result, research increasingly has taken user-centric aspects such as transparency and control of a RS into account, when assessing its quality. However, we argue that too little of this research has investigated the users' perception and understanding of RS in their entirety. In this paper, we explore the users' mental models of RS. More specifically, we followed the qualitative grounded theory methodology and conducted 10 semi-structured face-to-face interviews with typical and regular Netflix users. During interviews participants expressed high levels of uncertainty and confusion about the RS in Netflix. Consequently, we found a broad range of different mental models. Nevertheless, we also identified a general structure underlying all of these models, consisting of four steps: data acquisition, inference of user profile, comparison of user profiles or items, and generation of recommendations. Based on our findings, we discuss implications to design more transparent, controllable, and user friendly RS in the future.
Interactive Recommender Systems: A Survey of the State of the Art and Future Research Challenges and Opportunities

Chen He,Denis Parra,Katrien Verbert

DOI: https://doi.org/10.1016/j.eswa.2016.02.013

IF: 8.5

2016-01-01

Expert Systems with Applications

Abstract:Recommender systems have been researched extensively over the past decades. Whereas several algorithms have been developed and deployed in various application domains, recent research efforts are increasingly oriented towards the user experience of recommender systems. This research goes beyond accuracy of recommendation algorithms and focuses on various human factors that affect acceptance of recommendations, such as user satisfaction, trust, transparency and sense of control. In this paper, we present an interactive visualization framework that combines recommendation with visualization techniques to support human-recommender interaction. Then, we analyze existing interactive recommender systems along the dimensions of our framework, including our work. Based on our survey results, we present future research challenges and opportunities. (C) 2016 Elsevier Ltd. All rights reserved.
Usability Evaluation of a Knowledge Graph-Based Dementia Care Intelligent Recommender System: Mixed Methods Study

Minmin Leng,Yue Sun,Ce Li,Shuyu Han,Zhiwen Wang

DOI: https://doi.org/10.2196/45788

2023-09-26

Abstract:Background: Knowledge graph-based recommender systems offer the possibility of meeting the personalized needs of people with dementia and their caregivers. However, the usability of such a recommender system remains unknown. Objective: This study aimed to evaluate the usability of a knowledge graph-based dementia care intelligent recommender system (DCIRS). Methods: We used a convergent mixed methods design to conduct the usability evaluation, including the collection of quantitative and qualitative data. Participants were recruited through social media advertisements. After 2 weeks of DCIRS use, feedback was collected with the Computer System Usability Questionnaire and semistructured interviews. Descriptive statistics were used to describe sociodemographic characteristics and questionnaire scores. Qualitative data were analyzed systematically using inductive thematic analysis. Results: A total of 56 caregivers were recruited. Quantitative data suggested that the DCIRS was easy for caregivers to use, and the mean questionnaire score was 2.14. Qualitative data showed that caregivers generally believed that the content of the DCIRS was professional, easy to understand, and instructive, and could meet users' personalized needs; they were willing to continue to use it. However, the DCIRS also had some shortcomings. Functions that enable interactions between professionals and caregivers and that provide caregiver support and resource recommendations might be added to improve the system's usability. Conclusions: The recommender system provides a solution to meet the personalized needs of people with dementia and their caregivers and has the potential to substantially improve health outcomes. The next step will be to optimize and update the recommender system based on caregivers' suggestions and evaluate the effect of the application.
Concept -- An Evaluation Protocol on Conversational Recommender Systems with System-centric and User-centric Factors

Chen Huang,Peixin Qin,Yang Deng,Wenqiang Lei,Jiancheng Lv,Tat-Seng Chua

2024-05-06

Abstract:The conversational recommendation system (CRS) has been criticized regarding its user experience in real-world scenarios, despite recent significant progress achieved in academia. Existing evaluation protocols for CRS may prioritize system-centric factors such as effectiveness and fluency in conversation while neglecting user-centric aspects. Thus, we propose a new and inclusive evaluation protocol, Concept, which integrates both system- and user-centric factors. We conceptualise three key characteristics in representing such factors and further divide them into six primary abilities. To implement Concept, we adopt a LLM-based user simulator and evaluator with scoring rubrics that are tailored for each primary ability. Our protocol, Concept, serves a dual purpose. First, it provides an overview of the pros and cons in current CRS models. Second, it pinpoints the problem of low usability in the "omnipotent" ChatGPT and offers a comprehensive reference guide for evaluating CRS, thereby setting the foundation for CRS improvement.

Computation and Language,Artificial Intelligence
GroRec: A Group-Centric Intelligent Recommender System Integrating Social, Mobile and Big Data Technologies

Yin Zhang

DOI: https://doi.org/10.1109/tsc.2016.2592520

IF: 11.019

2016-01-01

IEEE Transactions on Services Computing

Abstract:In recent years, an extensive integration of cyber, physical and social spaces has been occurring. Cyber-Physical-Social Systems (CPSSs) have become the basic paradigm of evolution in the information industry, through which traditional computer science will evolve into cyber-physical-social computational science. Intelligent recommender systems, which are an important fundamental research topic in the CPSS field and one of the key techniques for the implementation of personalized and intelligent computing, have great significance in CPSS development. This paper proposes a group-centric recommender system in the CPSS domain, which consists of activity-oriented group discovery, the revision of rating data for improved accuracy, and group preference modeling that supports sufficient context mining from multiple sources. Through experiments, it is verified that the proposed recommender system is efficient, objective and accurate, thereby providing a strong foundation for personalized computing in the CPSS paradigm.
A Recommender System for Healthcare Based on Human-Centric Modeling

Zheyun Zhong,Yinsheng Li

DOI: https://doi.org/10.1109/icebe.2016.055

2016-01-01

Abstract:Personalized recommendation techniques are critical for a healthcare system to match patients with medical services. This work presented a human-centric user modeling approach and a recommendation system, which can work well at healthcare system with sparse operation data specially at the system's early stage. With human-centric modeling paradigm, both the user model and the item model are developed, and a matchmaking relationship are constructed between user intention and medical items. A user-to-user similarity matrix is developed to describe a user with other users. An item vector is developed to describe an item with its usage context. An experiment has been conducted on a healthcare prototype. The testing is encouraging and several proposed approaches and performance advantages are basically verified.
Recommender System:Up to Now

Yangyong ZHU,Jing SUN

DOI: https://doi.org/10.3778/j.issn.1673-9418.1412023

2015-01-01

Abstract:Recommender system is the product of cyber age today. There have been many achievements in research and application. This paper makes a comprehensive survey of the recommender system. It proposes three research phases, and points out the milestone events in each stage of recommender system development. In the age of big data, exploiting recommendation in the perspective of data, this paper classifies the recommender system into seven main classes according to the different data used in recommendation, and analyzes and comments the recommended models used in each classification and their advantages and disadvantages. Exploiting big data in the perspective of recom-mendation, this paper proposes that making recommendation based on big data is one of the promising research direc-tions. Finally, this paper compares the evaluation metrics of recommendation, and gives future research directions.
A Survey on Data-Centric Recommender Systems

Riwei Lai,Rui Chen,Chi Zhang

2024-05-28

Abstract:Recommender systems (RSs) have become an essential tool for mitigating information overload in a range of real-world applications. Recent trends in RSs have revealed a major paradigm shift, moving the spotlight from model-centric innovations to data-centric efforts (e.g., improving data quality and quantity). This evolution has given rise to the concept of data-centric recommender systems (Data-Centric RSs), marking a significant development in the field. This survey provides the first systematic overview of Data-Centric RSs, covering 1) the foundational concepts of recommendation data and Data-Centric RSs; 2) three primary issues of recommendation data; 3) recent research developed to address these issues; and 4) several potential future directions of Data-Centric RSs.

Information Retrieval
The Design and Development of the Virtual Learning Community for Teaching Resources Personalized Recommendation.

Bo Song,Haihui Wu,Xiaomei Li,Liyan Guo,Chang Liu

DOI: https://doi.org/10.1007/978-3-319-61833-3_12

2017-01-01

Abstract:With the integration of advanced concepts and technologies, such as Web2.0, Web3.0 and cloud computing, the teaching resources of the virtual learning community are constantly enriched and expanded to bring diversification and autonomy to the users. It also to different backgrounds and different preferences of the user's teaching resources required for the inconvenience in a large number of teaching resources can not accurately choose their own teaching resources. In order to more effectively recommend the products of interest to users, this paper uses Java EE 5 technology to achieve a user interface with good interactivity. At the same time, an ALS recommendation algorithm based on Spark is proposed, and the performance and effect evaluation of the system are made in the virtual learning community.

User-Centric Design and Evaluation of Online Interactive Recommender Systems

A Novel Framework To Process The Quantity And Quality Of User Behavior Data In Recommender Systems

Optimizing the Design of Recommendation Systems.

RECENT ADVANCES IN PERSONAL RECOMMENDER SYSTEMS

Personal Recommendation in User-Object Networks

User Interest Dynamics on Personalized Recommendation

Evaluating Conversational Recommender Systems: A Landscape of Research

Uncovering the Information Core in Recommender Systems

Human-Centered Recommender Systems: Origins, Advances, Challenges, and Opportunities

How good your recommender system is? A survey on evaluations in recommendation

Analysis and Design of a Personalized Recommendation System Based on a Dynamic User Interest Model

Product and User Dependent Social Network Models for Recommender Systems.

Exploring Mental Models for Transparent and Controllable Recommender Systems: A Qualitative Study

Interactive Recommender Systems: A Survey of the State of the Art and Future Research Challenges and Opportunities

Usability Evaluation of a Knowledge Graph-Based Dementia Care Intelligent Recommender System: Mixed Methods Study

Concept -- An Evaluation Protocol on Conversational Recommender Systems with System-centric and User-centric Factors

GroRec: A Group-Centric Intelligent Recommender System Integrating Social, Mobile and Big Data Technologies

A Recommender System for Healthcare Based on Human-Centric Modeling

Recommender System:Up to Now

A Survey on Data-Centric Recommender Systems

The Design and Development of the Virtual Learning Community for Teaching Resources Personalized Recommendation.