Learn To Rank: Automatic Helpfulness Analysis Of Online Product Reviews

Jian Jin,Ying Liu,Jenny A. Harding,Richard Y. K. Fung,Han Tong Loh

2010-01-01

Abstract:Online reviews are the valuable voice of the customer. Product designers can gain insights into their customers and products through analyzing such review comments, and eventually, improve their products accordingly. However, the sheer amount of reviews, their disparate locations and the inherent ambiguity of human language have greatly challenged designers. In this paper, the focus is on how to automatically learn, rank and predict the usefulness of customer reviews, based on the review content and other factors that are available in the online context. The ultimate goal of this study is to intelligently distill quality reviews that designers can trust and rely on. We begin with a definition of helpfulness and then propose an automatic ranking approach based on the prediction of the helpfulness of the online reviews. Our approach directly utilizes key characteristic information, e.g. number of product features and average review length, embedded in the reviews to estimate their contribution towards a judgment of helpfulness. Our experimental study, using a very large amount of real world data trawled from Amazon, shows some very promising results. The predictions, using Random Forest, a decision tree approach, have delivered a performance of nearly 70% in terms of the F-1 value.