Abstract:It is important to quantify the differences in returns to skills using the online job advertisements data, which have attracted great interest in both labor economics and statistics fields. In this paper, we study the relationship between the posted salary and the job requirements in online labor markets. There are two challenges to deal with. First, the posted salary is always presented in an interval-valued form, for example, 5k-10k yuan per month. Simply taking the mid-point or the lower bound as the alternative for salary may result in biased estimators. Second, the number of the potential skill words as predictors generated from the job advertisements by word segmentation is very large and many of them may not contribute to the salary. To this end, we propose a new feature screening method, Absolute Distribution Difference Sure Independence Screening (ADD-SIS), to select important skill words for the interval-valued response. The marginal utility for feature screening is based on the difference of estimated distribution functions via nonparametric maximum likelihood estimation, which sufficiently uses the interval information. It is model-free and robust to outliers. Numerical simulations show that the new method using the interval information is more efficient to select important predictors than the methods only based on the single points of the intervals. In the real data application, we study the text data of job advertisements for data scientists and data analysts in a major China's online job posting website, and explore the important skill words for the salary. We find that the skill words like optimization, long short-term memory (LSTM), convolutional neural networks (CNN), collaborative filtering, are positively correlated with the salary while the words like Excel, Office, data collection, may negatively contribute to the salary.

A supervised weeding method to cluster high dimensional predictors with application to job market analysis

Career Choice Prediction Based on Campus Big Data-Mining the Potential Behavior of College Students

Cost-sensitive Regression Learning on Small Dataset Through Intra-Cluster Product Favoured Feature Selection

A Two-Stage Variable Selection Approach for Correlated High Dimensional Predictors

End-to-End Solution with Clustering Method for Attrition Analysis

Grouping predictors via network-wide metrics

Feature Screening for Interval-Valued Response with Application to Study Association between Posted Salary and Required Skills

Feature Selection for Learning to Predict Outcomes of Compute Cluster Jobs with Application to Decision Support

A Feature Selection Framework Based on Supervised Data Clustering

Multivariate Cluster Analysis on College Students' Employment Evaluation

Simultaneous Estimation of Number of Clusters and Feature Sparsity in Clustering High-Dimensional Data

A Self-Organizing Method for Predictive Modeling with Highly-Redundant Variables

Unsupervised Feature Selection with Joint Clustering Analysis.

A Method for Predicting the Impact of Labor Education at Institutions of Higher Education Based on a Two-Stage Clustering Algorithm

Job and Candidate Recommendation with Big Data Support: A Contextual Online Learning Approach

Clustering-Guided Sparse Structural Learning for Unsupervised Feature Selection

WITHDRAWN: College student employment data platform based on FPGA and machine learning

Statistica Sinica Preprint No : SS-2018-0032 Title Nonparametric Cluster Analysis on Multiple Outcomes of Longitudinal Data

Efficient and Doubly-Robust Methods for Variable Selection and Parameter Estimation in Longitudinal Data Analysis

Exploiting Word Cluster Information for Unsupervised Feature Selection

Nonparametric Cluster Analysis on Multiple Outcomes of Longitudinal Data