Abstract:Profiling Web users is a fundamental issue for Web mining and social network analysis. Its basic tasks include extracting basic information, mining user preferences, and inferring user demographics (Tang et al. in ACM Trans Knowl Discov Data 5(1):2:1–2:44, 2010). Although methodologies for handling the three tasks are different, they all usually contain two stages: first identify relevant pages (data) of a user and then use machine learning models (e.g., SVM, CRFs, or DL) to extract/mine/infer profile attributes from each page. The methods were successful in the traditional Web, but are facing more and more challenges with the rapid evolution of the Web each persons information is distributed over the Web and is changing dynamically. As a result, available data for a user on the Web is redundant, and some sources may be out-of-date or incorrect. The traditional two-stage method suffers from data inconsistency and error propagation between the two stages. In this paper, we revisit the problem of Web user profiling in the big data era and propose a simple but very effective approach, referred to as MagicFG, for profiling Web users by leveraging the power of big data. To avoid error propagation, the approach processes all the extracting/mining/inferring subtasks in one unified framework. To improve the profiling performance, we present the concept of contextual credibility. The proposed framework also supports the incorporation of human knowledge. It defines human knowledge as Markov logics statements and formalizes them into a factor graph model. The MagicFG method has been deployed in an online system AMiner.org for profiling millions of researchers—e.g., extracting E-mail, inferring Gender, and mining research interests. Our empirical study in the real system shows that the proposed method offers significantly improved (+ 4–6%; \(p\ll 0.01\), t test) profiling performance in comparison with several baseline methods using rules, classification, and sequential labeling.

Non-Negative Matrix for Mining Typical User Profiles Factorization

Local Nonnegative Matrix Factorization for Mining Typical User Session Profile

Matrix dimensionality reduction for mining typical user profiles

Matrix Dimensionality Reduction for Mining Web Logs

Obtaining Profiles Based on Localized Non-Negative Matrix Factorization

Feature Extraction Via Multi-View Non-Negative Matrix Factorization with Local Graph Regularization.

Nonnegative Matrix Factorization on Orthogonal Subspace

Distributed Nonnegative Matrix Factorization for Web-Scale Dyadic Data Analysis on Mapreduce.

Collaborative filtering based on nonnegative/binary matrix factorization

Non-negative Matrix Factorization on Manifold

Dynamic Semantic Clustering Approach For Web User Interest

Feature Weighted Non-negative Matrix Factorization

Locality Preserving Nonnegative Matrix Factorization.

Towards a Fairer Non-negative Matrix Factorization

Factor-Bounded Nonnegative Matrix Factorization

Discriminative Concept Factorization for Data Representation

Profiling Web Users Using Big Data

Information Filtering Via Collaborative User Clustering Modeling.

Non-negative Matrix Factorization with Pairwise Constraints and Graph Laplacian

Non-negative Matrix-Set Factorization

Nonnegative Matrix Factorization with Integrated Graph and Feature Learning