Abstract:Profiling Web users is a fundamental issue for Web mining and social network analysis. Its basic tasks include extracting basic information, mining user preferences, and inferring user demographics (Tang et al. in ACM Trans Knowl Discov Data 5(1):2:1–2:44, 2010). Although methodologies for handling the three tasks are different, they all usually contain two stages: first identify relevant pages (data) of a user and then use machine learning models (e.g., SVM, CRFs, or DL) to extract/mine/infer profile attributes from each page. The methods were successful in the traditional Web, but are facing more and more challenges with the rapid evolution of the Web each persons information is distributed over the Web and is changing dynamically. As a result, available data for a user on the Web is redundant, and some sources may be out-of-date or incorrect. The traditional two-stage method suffers from data inconsistency and error propagation between the two stages. In this paper, we revisit the problem of Web user profiling in the big data era and propose a simple but very effective approach, referred to as MagicFG, for profiling Web users by leveraging the power of big data. To avoid error propagation, the approach processes all the extracting/mining/inferring subtasks in one unified framework. To improve the profiling performance, we present the concept of contextual credibility. The proposed framework also supports the incorporation of human knowledge. It defines human knowledge as Markov logics statements and formalizes them into a factor graph model. The MagicFG method has been deployed in an online system AMiner.org for profiling millions of researchers—e.g., extracting E-mail, inferring Gender, and mining research interests. Our empirical study in the real system shows that the proposed method offers significantly improved (+ 4–6%; \(p\ll 0.01\), t test) profiling performance in comparison with several baseline methods using rules, classification, and sequential labeling.

Matrix dimensionality reduction for mining typical user profiles

Non-Negative Matrix for Mining Typical User Profiles Factorization

Matrix Dimensionality Reduction for Mining Web Logs

Local Nonnegative Matrix Factorization for Mining Typical User Session Profile

Using Visualization to Improve Clustering Analysis on Heterogeneous Information Network.

A Dynamic Topic Model and Matrix Factorization-Based Travel Recommendation Method Exploiting Ubiquitous Data.

Information Filtering Via Collaborative User Clustering Modeling.

A Fuzzy Approach for Feature Evaluation and Dimensionality Reduction to Improve the Quality of Web Usage Mining Results

Dynamic Semantic Clustering Approach For Web User Interest

A framework for mining lifestyle profiles through multi-dimensional and high-order mobility feature clustering

Discovery of Web Usage Profiles Using Various Clustering Techniques

An Effective Algorithm For Dimensional Reduction In Collaborative Filtering

Matrix clustering-based collaborative filtering algorithm

Effectively clustering researchers in scientific collaboration networks: case study on ResearchGate

Profiling Web Users Using Big Data

Unified Matrix Factorization with Dynamic Multi-view Clustering

Learning From Hidden Traits: Joint Factor Analysis and Latent Clustering

Automatic Spectral Clustering of User Sessions Based on the Similarity of Global Alignment

A New Similarity in Clustering Through Users' Interest and Social Relationship

Cross-Domain Recommendation Via User-Clustering and Multidimensional Information Fusion

Revisiting Dimensionality Reduction Techniques for Visual Cluster Analysis: An Empirical Study