Abstract:Profiling Web users is a fundamental issue for Web mining and social network analysis. Its basic tasks include extracting basic information, mining user preferences, and inferring user demographics (Tang et al. in ACM Trans Knowl Discov Data 5(1):2:1–2:44, 2010). Although methodologies for handling the three tasks are different, they all usually contain two stages: first identify relevant pages (data) of a user and then use machine learning models (e.g., SVM, CRFs, or DL) to extract/mine/infer profile attributes from each page. The methods were successful in the traditional Web, but are facing more and more challenges with the rapid evolution of the Web each persons information is distributed over the Web and is changing dynamically. As a result, available data for a user on the Web is redundant, and some sources may be out-of-date or incorrect. The traditional two-stage method suffers from data inconsistency and error propagation between the two stages. In this paper, we revisit the problem of Web user profiling in the big data era and propose a simple but very effective approach, referred to as MagicFG, for profiling Web users by leveraging the power of big data. To avoid error propagation, the approach processes all the extracting/mining/inferring subtasks in one unified framework. To improve the profiling performance, we present the concept of contextual credibility. The proposed framework also supports the incorporation of human knowledge. It defines human knowledge as Markov logics statements and formalizes them into a factor graph model. The MagicFG method has been deployed in an online system AMiner.org for profiling millions of researchers—e.g., extracting E-mail, inferring Gender, and mining research interests. Our empirical study in the real system shows that the proposed method offers significantly improved (+ 4–6%; \(p\ll 0.01\), t test) profiling performance in comparison with several baseline methods using rules, classification, and sequential labeling.

A Comprehensive Survey of Recent Research on Profile Data Analysis

Ieee Access Special Section Editorial: Advanced Data Analytics For Large-Scale Complex Data Environments

Multiple Profiles Sensor-Based Monitoring and Anomaly Detection

Profile Inference from Heterogeneous Data - Fundamentals and New Trends.

A Survey on Sampling and Profiling over Big Data (Technical Report)

User Modeling and User Profiling: A Comprehensive Survey

In-profile Monitoring for Cluster-Correlated Data in Advanced Manufacturing System

Social User Profiling: A Social-Aware Topic Modeling Perspective

Exploratory Analysis for Big Social Data Using Deep Network

Monitoring and prediction of big process data with deep latent variable models and parallel computing

Weakly Correlated Profile Monitoring Based on Sparse Multi-Channel Functional Principal Component Analysis

A Survey of Recent 3D Scene Analysis and Processing Methods

Advanced Persistent Threat intelligent profiling technique: A survey

A Combination Approach to Web User Profiling

Functional State-Space Model for Multi-Channel Autoregressive Profiles with Application in Advanced Manufacturing

Advanced Data Collection and Analysis in Data-Driven Manufacturing Process

Profiling Web Users Using Big Data

Investigations on Interpretable Nuclear-Grade Hydraulic Damper Monitoring and Diagnosis from Multi-Source High-Dimensional Profile Data

Data-Based Techniques Focused on Modern Industry: an Overview

A Network Datagram and Big Data Based Research on Method of User Profile

Multilinear and Nonlinear Generalizations of Partial Least Squares: an Overview of Recent Advances