Abstract:Rationale refers to making human judgments, sets of reasons, or intentions to explain a particular decision. Nowadays, crowd-users argue and justify their decisions on social media platforms about market-driven software applications, thus generating a software rationale. Such rationale information can be of pivotal importance for the software and requirements engineers to enhance the performance of existing software applications by revealing end-users tactic knowledge to improve software designing and development decision-making. For this purpose, we proposed an automated approach to capture and analyze end-user reviews containing rationale information, focusing on low-rating applications in the amazon store using Natural Language Processing (NLP) and supervised machine learning (ML) classification methods. In the literature, high-rating applications have been emphasized while ignoring low-rating software application that causes potential biasness. Therefore, we examined 59 comparatively low-ranked market-based software applications from the Amazon app store covering various software categories to capture and identify crowd-users justifications. Next, using a developed grounded theory and content analysis approach, we studied and recorded how crowd-users analyze and explain their rationale based on issues encountered, attacking or supporting arguments registered, and updating or uninstalling software applications. Also, to achieve the best results, an experimental study is conducted by comparing various ML and deep learning (DL) algorithms, i.e., MNB, LR, RF, MLP, KNN, AdaBoost, Voting, LSTM, and BILSTM classifiers on the end-users rationale data set by preprocessing the input data, applying feature engineering, balancing the data set, and then training and testing the ML algorithms with a standard cross-validation approach. We obtained satisfactory results with MLP, voting, LSTM, and RF Classifiers, having 93%, 93%, 91%, and 90% average accuracy, respectively. Also, we plot the ROC curves for the high-performing ML and DL classifiers to identify and capture classifiers yielding the best performance with an under-sampling or oversampling balancing approach. The proposed research approach outerperforms the existing rationale approaches with better Precision, Recall, and F-measure values. Additionally, the paper discusses various aspects of the proposed approach by extending it in multiple directions to be utilized by software developers and vendors to help improve the performance of existing software applications.

A NLP Approach to "Review Bombing" in Metacritic PC Videogames User Ratings

Analyzing and Detecting Adversarial Spam on a Large-scale Online APP Review System.

Recommending Code Reviewers for Proprietary Software Projects: A Large Scale Study

The Wisdom of the Gaming Crowd

Sex, drugs, and violence

Toward Human-Like Evaluation for Natural Language Generation with Error Analysis

Analysis of Systems' Performance in Natural Language Processing Competitions

Length and sentiment analysis of reviews about top-level video game genres on the steam platform

Unmasking Falsehoods in Reviews: An Exploration of NLP Techniques

Game Reviews Reviewed: A Game Designer’s Perspective on AI-generated Game Review Analyses

Exploring and mining rationale information for low-rating software applications

Automated Crowdturfing Attacks and Defenses in Online Review Systems

Can We Automate Scientific Reviewing?

MONITORING AND REMOVAL OF FAKE PRODUCT REVIEW USING MACHINE LEARNING (ML)

Improving Code Reviewer Recommendation: Accuracy, Latency, Workload, and Bystanders

Peer review analyze: A novel benchmark resource for computational analysis of peer reviews

An empirical study of game reviews on the Steam platform

Advancements and Methodologies in Natural Language Processing and Machine Learning: A Comprehensive Review

Explaining and Predicting Helpfulness and Funniness of Online Reviews on the Steam Platform

LLMs Assist NLP Researchers: Critique Paper (Meta-)Reviewing