Computational Job Market Analysis with Natural Language Processing

Mike Zhang

2024-04-29

Abstract:[Abridged Abstract]

Computation and Language

What problem does this paper attempt to address?

Deep Learning-based Computational Job Market Analysis: A Survey on Skill Extraction and Classification from Job Postings

Elena Senger,Mike Zhang,Rob van der Goot,Barbara Plank

2024-02-08

Abstract:Recent years have brought significant advances to Natural Language Processing (NLP), which enabled fast progress in the field of computational job market analysis. Core tasks in this application domain are skill extraction and classification from job postings. Because of its quick growth and its interdisciplinary nature, there is no exhaustive assessment of this emerging field. This survey aims to fill this gap by providing a comprehensive overview of deep learning methodologies, datasets, and terminologies specific to NLP-driven skill extraction and classification. Our comprehensive cataloging of publicly available datasets addresses the lack of consolidated information on dataset creation and characteristics. Finally, the focus on terminology addresses the current lack of consistent definitions for important concepts, such as hard and soft skills, and terms relating to skill extraction and classification.

Computer Science
A Big Data Analysis of Job Position Status Based on Natural Language Processing

Qi Liu

DOI: https://doi.org/10.1088/1755-1315/693/1/012026

2021-03-01

IOP Conference Series: Earth and Environmental Science

Abstract:Abstract With the development of the Internet and computer technology, news on the Internet has gradually been able to influence the development of the economy. In some special periods, with the development of major events, the economy and finance have undergone similar development and changes. This paper proposes a simplified natural language processing method for multi-grained text. First, the news text data is classified according to sentiment, and the correlation between corporate loans and job retention ability is further analysed. The experimental results show that under certain conditions, the Loans issued to SMEs are more conducive to retaining the number of jobs than large enterprises.
Development and Evaluation of Task-Specific NLP Framework in China.

Caixia Ge,Yinsheng Zhang,Zhenzhen Huang,Zheng Jia,Meizhi Ju,Huilong Duan,Haomin Li

DOI: https://doi.org/10.3233/978-1-61499-564-7-1031

2015-01-01

Abstract:Natural language processing (NLP) has been designed to convert narrative text into structured data. Although some general NLP architectures have been developed, a task-specific NLP framework to facilitate the effective use of data is still a challenge in lexical resource limited regions, such as China. The purpose of this study is to design and develop a task-specific NLP framework to extract targeted information from particular documents by adopting dedicated algorithms on current limited lexical resources. In this framework, a shared and evolving ontology mechanism was designed. The result has shown that such a free text driven platform will accelerate the NLP technology acceptance in China.
Understanding and Modeling Job Marketplace with Pretrained Language Models

Yaochen Zhu,Liang Wu,Binchi Zhang,Song Wang,Qi Guo,Liangjie Hong,Luke Simon,Jundong Li

2024-08-08

Abstract:Job marketplace is a heterogeneous graph composed of interactions among members (job-seekers), companies, and jobs. Understanding and modeling job marketplace can benefit both job seekers and employers, ultimately contributing to the greater good of the society. However, existing graph neural network (GNN)-based methods have shallow understandings of the associated textual features and heterogeneous relations. To address the above challenges, we propose PLM4Job, a job marketplace foundation model that tightly couples pretrained language models (PLM) with job market graph, aiming to fully utilize the pretrained knowledge and reasoning ability to model member/job textual features as well as various member-job relations simultaneously. In the pretraining phase, we propose a heterogeneous ego-graph-based prompting strategy to model and aggregate member/job textual features based on the topological structure around the target member/job node, where entity type embeddings and graph positional embeddings are introduced accordingly to model different entities and their heterogeneous relations. Meanwhile, a proximity-aware attention alignment strategy is designed to dynamically adjust the attention of the PLM on ego-graph node tokens in the prompt, such that the attention can be better aligned with job marketplace semantics. Extensive experiments at LinkedIn demonstrate the effectiveness of PLM4Job.

Information Retrieval
Identifying Labor Market Competitors with Machine Learning Based on Maimai Platform

Yu Zheng,Yonghong Long,Honggang Fan

DOI: https://doi.org/10.1080/08839514.2022.2064047

IF: 2.777

2022-04-18

Applied Artificial Intelligence

Abstract:The demand for skilled labor has increased dramatically in the current knowledge-based economy, which is characterized by the growing intensity in labor market competition between firms. Therefore, it would be of special interest to identify future labor market competitors. At present, with the vast amount of textual data, the existing study focuses on constructing the human capital overlap and product overlap metrics with the text data as predictors to predict the labor market competition in the United States. Based on these metrics, this paper experiments with machine learning methods to predict Chinese labor market competition with Chinese text data. Furthermore, sentiment analysis is becoming popular and it has been used in a wide variety of fields. However, due to lack of data, there are few existing studies using sentiment analysis approach of firms’ online reviews. In response to this research gap, this paper constructs the sentiment analysis metric by mining the emotional content expressed in the firms’ online reviews on Maimai’s platform. The results show that our proposed metrics have superior predictive power over conventional measures and highlight the predictive utility of proposed sentiment analysis metric. Moreover, the nuanced two-dimensional competition analysis gives some interesting results.

computer science, artificial intelligence,engineering, electrical & electronic
Occupational models from 42 million unstructured job postings

Nile Dixon,Marcelle Goggins,Ethan Ho,Mark Howison,Joe Long,Emma Northcott,Karen Shen,Carrie Yeats

DOI: https://doi.org/10.1016/j.patter.2023.100757

2023-05-22

Abstract:Structuring jobs into occupations is the first step for analysis tasks in many fields of research, including economics and public health, as well as for practical applications like matching job seekers to available jobs. We present a data resource, derived with natural language processing techniques from over 42 million unstructured job postings in the National Labor Exchange, that empirically models the associations between occupation codes (estimated initially by the Standardized Occupation Coding for Computer-assisted Epidemiological Research method), skill keywords, job titles, and full-text job descriptions in the United States during the years 2019 and 2021. We model the probability that a job title is associated with an occupation code and that a job description is associated with skill keywords and occupation codes. Our models are openly available in the sockit python package, which can assign occupation codes to job titles, parse skills from and assign occupation codes to job postings and resumes, and estimate occupational similarity among job postings, resumes, and occupation codes.
Resume Parser Analysis Using Machine Learning and Natural Language Processing

Pratham Rasal

DOI: https://doi.org/10.22214/ijraset.2023.52202

2023-05-31

International Journal for Research in Applied Science and Engineering Technology

Abstract:Abstract: With the rise of online job application processes, submitting a resume has become easier than ever before. As a result, a larger number of individuals are impacted by this change. Many organizations still accept resumes by mail, which can cause challenges for their human resources departments. Sorting through a large number of applications to find the most suitable candidates can be a time-consuming process for these agencies. Job applicants submit resumes in a wide range of formats, including various fonts, font sizes, colors, and other design elements. Human resources departments are responsible for reviewing each application and selecting the most qualified candidate for the job. My suggestion is to incorporate natural language processing techniques into the project's parser to assist the human resources department or recruiting manager in analyzing the information provided in resumes. This approach can involve using keyword matching and other natural language processing methods to identify the most suitable candidates and obtain the most effective resumes. By doing so, the organization can improve its recruitment process and identify the best candidates for the job.
Data-driven HR - Résumé Analysis Based on Natural Language Processing and Machine Learning

Tim Zimmermann,Leo Kotschenreuther,Karsten Schmidt

DOI: https://doi.org/10.48550/arXiv.1606.05611

2016-06-22

Abstract:Recruiters usually spend less than a minute looking at each résumé when deciding whether it's worth continuing the recruitment process with the candidate. Recruiters focus on keywords, and it's almost impossible to guarantee a fair process of candidate selection. The main scope of this paper is to tackle this issue by introducing a data-driven approach that shows how to process résumés automatically and give recruiters more time to only examine promising candidates. Furthermore, we show how to leverage Machine Learning and Natural Language Processing in order to extract all required information from the résumés. Once the information is extracted, a ranking score is calculated. The score describes how well the candidates fit based on their education, work experience and skills. Later this paper illustrates a prototype application that shows how this novel approach can increase the productivity of recruiters. The application enables them to filter and rank candidates based on predefined job descriptions. Guided by the ranking, recruiters can get deeper insights from candidate profiles and validate why and how the application ranked them. This application shows how to improve the hiring process by giving an unbiased hiring decision support.

Computation and Language,Artificial Intelligence
NNOSE: Nearest Neighbor Occupational Skill Extraction

Mike Zhang,Rob van der Goot,Min-Yen Kan,Barbara Plank

2024-01-30

Abstract:The labor market is changing rapidly, prompting increased interest in the automatic extraction of occupational skills from text. With the advent of English benchmark job description datasets, there is a need for systems that handle their diversity well. We tackle the complexity in occupational skill datasets tasks -- combining and leveraging multiple datasets for skill extraction, to identify rarely observed skills within a dataset, and overcoming the scarcity of skills across datasets. In particular, we investigate the retrieval-augmentation of language models, employing an external datastore for retrieving similar skills in a dataset-unifying manner. Our proposed method, \textbf{N}earest \textbf{N}eighbor \textbf{O}ccupational \textbf{S}kill \textbf{E}xtraction (NNOSE) effectively leverages multiple datasets by retrieving neighboring skills from other datasets in the datastore. This improves skill extraction \emph{without} additional fine-tuning. Crucially, we observe a performance gain in predicting infrequent patterns, with substantial gains of up to 30\% span-F1 in cross-dataset settings.

Computation and Language
Skill Extraction from Job Postings using Weak Supervision

Mike Zhang,Kristian Nørgaard Jensen,Rob van der Goot,Barbara Plank

DOI: https://doi.org/10.48550/arXiv.2209.08071

2022-09-17

Abstract:Aggregated data obtained from job postings provide powerful insights into labor market demands, and emerging skills, and aid job matching. However, most extraction approaches are supervised and thus need costly and time-consuming annotation. To overcome this, we propose Skill Extraction with Weak Supervision. We leverage the European Skills, Competences, Qualifications and Occupations taxonomy to find similar skills in job ads via latent representations. The method shows a strong positive signal, outperforming baselines based on token-level and syntactic patterns.

Computation and Language
Resume Analysis Using Machine Learning and Natural Language Processing

Sakshi, Prakash,

DOI: https://doi.org/10.55041/ijsrem31638

2024-04-25

INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT

Abstract:The recent recruitment process has changed from manually to computerized. The majority of resume analysis is carried out utilizing Natural Language Processing and Machine learning. It deals with selecting the best candidates from an enormous pool of applicants who have the necessary skills for a certain job description.By leveraging these NLP capabilities provided by NLTK, resume analysis systems can automate the extraction, classification, and evaluation of information from resumes, enabling recruiters to efficiently screen and evaluate candidates based on their qualifications, skills, experience, and other relevant factors. Keywords - Cosine Similarity, Machine Learning, TDF- IDF, NLP-Natural Language Processing.
Machine Learned Resume-Job Matching Solution

Yiou Lin,Hang Lei,Prince Clement Addo,Xiaoyu Li

DOI: https://doi.org/10.48550/arXiv.1607.07657

2016-07-26

Computation and Language

Abstract:Job search through online matching engines nowadays are very prominent and beneficial to both job seekers and employers. But the solutions of traditional engines without understanding the semantic meanings of different resumes have not kept pace with the incredible changes in machine learning techniques and computing capability. These solutions are usually driven by manual rules and predefined weights of keywords which lead to an inefficient and frustrating search experience. To this end, we present a machine learned solution with rich features and deep learning methods. Our solution includes three configurable modules that can be plugged with little restrictions. Namely, unsupervised feature extraction, base classifiers training and ensemble method learning. In our solution, rather than using manual rules, machine learned methods to automatically detect the semantic similarity of positions are proposed. Then four competitive "shallow" estimators and "deep" estimators are selected. Finally, ensemble methods to bag these estimators and aggregate their individual predictions to form a final prediction are verified. Experimental results of over 47 thousand resumes show that our solution can significantly improve the predication precision current position, salary, educational background and company scale.
ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain

Mike Zhang,Rob van der Goot,Barbara Plank

2023-05-20

Abstract:The increasing number of benchmarks for Natural Language Processing (NLP) tasks in the computational job market domain highlights the demand for methods that can handle job-related tasks such as skill extraction, skill classification, job title classification, and de-identification. While some approaches have been developed that are specific to the job market domain, there is a lack of generalized, multilingual models and benchmarks for these tasks. In this study, we introduce a language model called ESCOXLM-R, based on XLM-R, which uses domain-adaptive pre-training on the European Skills, Competences, Qualifications and Occupations (ESCO) taxonomy, covering 27 languages. The pre-training objectives for ESCOXLM-R include dynamic masked language modeling and a novel additional objective for inducing multilingual taxonomical ESCO relations. We comprehensively evaluate the performance of ESCOXLM-R on 6 sequence labeling and 3 classification tasks in 4 languages and find that it achieves state-of-the-art results on 6 out of 9 datasets. Our analysis reveals that ESCOXLM-R performs better on short spans and outperforms XLM-R on entity-level and surface-level span-F1, likely due to ESCO containing short skill and occupation titles, and encoding information on the entity-level.

Computation and Language
Automated staff assignment for building maintenance using natural language processing

Yunjeong Mo,Dong Zhao,Jing Du,Matt Syal,Azizan Aziz,Heng Li

DOI: https://doi.org/10.1016/j.autcon.2020.103150

IF: 10.3

2020-05-01

Automation in Construction

Abstract:Staff assignment is the decision-making to determine appropriate workforce with required skills to perform a specific task. Staff assignment is critical to success of construction projects, especially when responding to routine requests such as the change order and building service. However, the effectiveness is low due to manual processing by the management personnel. To improve the productivity of staff assignment, this paper creates a machine learning model that reads service request texts and automatically assigns workforce and priority through the technique of natural language processing (NLP). The dataset used for modeling in this study contains 82,106 building maintenance records for a 3-year period from over 60 buildings on a university campus. The results show a 77% accuracy for predicting workforce and an 88% accuracy for predicting priority, indicating a considerably high performance for multiclass and binary classifications. Different from existing studies, the NLP model highlights the value of stop-words and punctuation in learning service request texts. The NLP model presented in this study provides a solution for staff assignment and offers a piece of the puzzle to the information system automation in the construction industry. This study has an immediate implication for building maintenance; and, in the long term, contributes to human-building interactions in smart buildings by connecting human feedback to building control systems.

construction & building technology,engineering, civil
RESUME SUMMARIZER AND JOB DESCRIPTION MATCHER USING NATURAL LANGUAGE PROCESSING AND SPACY

Sai Tarun Boddu,,Sujeeth Desu,Sreekanth Puli

DOI: https://doi.org/10.55041/ijsrem26647

2023-11-01

INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT

Abstract:In the HR recruitment process, one pivotal stage is "Summarizing & Screening." At this juncture, recruiters grapple with the laborious task of manually sifting through a myriad of resumes to shortlist suitable candidates. This project aims to streamline and automate this process using two innovative tools. Firstly, the "Resume Summarizer" leverages advanced natural language processing techniques to swiftly extract pertinent details from candidates' resumes, offering a concise summary that underscores their skills, experience, and qualifications. This summary can be tailored to match specific job requirements, facilitating recruiters in the comparative evaluation of multiple candidates. Secondly, the "Job Description Matcher" assists recruiters in expeditiously and accurately aligning job descriptions with candidate resumes. By employing natural language processing, this tool identifies keywords and phrases that align with the job opening's prerequisites, generating a compatibility score to rank resumes according to their alignment with the job description. This innovation greatly expedites candidate selection and enhances recruiters' efficiency. Importantly, this project employs a unique approach, incorporating regular expressions to extract essential information from resumes, setting it apart from existing NLP-based models and systems. Keywords: Natural Language Processing, Text Analysis, Data Extraction, Keyword matching, Regular Expressions, Optical Character Recognition (OCR), N-Gram Model, Name Entity Recognition (NER), Unsupervised NER.
The Future of ChatGPT-enabled Labor Market: A Preliminary Study in China

Lan Chen,Xi Chen,Shiyu Wu,Yaqi Yang,Meng Chang,Hengshu Zhu

2023-11-04

Abstract:As a phenomenal large language model, ChatGPT has achieved unparalleled success in various real-world tasks and increasingly plays an important role in our daily lives and work. However, extensive concerns are also raised about the potential ethical issues, especially about whether ChatGPT-like artificial general intelligence (AGI) will replace human jobs. To this end, in this paper, we introduce a preliminary data-driven study on the future of ChatGPT-enabled labor market from the view of Human-AI Symbiosis instead of Human-AI Confrontation. To be specific, we first conduct an in-depth analysis of large-scale job posting data in BOSS Zhipin, the largest online recruitment platform in China. The results indicate that about 28% of occupations in the current labor market require ChatGPT-related skills. Furthermore, based on a large-scale occupation-centered knowledge graph, we develop a semantic information enhanced collaborative filtering algorithm to predict the future occupation-skill relations in the labor market. As a result, we find that additional 45% occupations in the future will require ChatGPT-related skills. In particular, industries related to technology, products, and operations are expected to have higher proficiency requirements for ChatGPT-related skills, while the manufacturing, services, education, and health science related industries will have lower requirements for ChatGPT-related skills.

Computers and Society,Artificial Intelligence
Comparative Analysis of Encoder-Based NER and Large Language Models for Skill Extraction from Russian Job Vacancies

Nikita Matkin,Aleksei Smirnov,Mikhail Usanin,Egor Ivanov,Kirill Sobyanin,Sofiia Paklina,Petr Parshakov

2024-09-15

Abstract:The labor market is undergoing rapid changes, with increasing demands on job seekers and a surge in job openings. Identifying essential skills and competencies from job descriptions is challenging due to varying employer requirements and the omission of key skills. This study addresses these challenges by comparing traditional Named Entity Recognition (NER) methods based on encoders with Large Language Models (LLMs) for extracting skills from Russian job vacancies. Using a labeled dataset of 4,000 job vacancies for training and 1,472 for testing, the performance of both approaches is evaluated. Results indicate that traditional NER models, especially DeepPavlov RuBERT NER tuned, outperform LLMs across various metrics including accuracy, precision, recall, and inference time. The findings suggest that traditional NER models provide more effective and efficient solutions for skill extraction, enhancing job requirement clarity and aiding job seekers in aligning their qualifications with employer expectations. This research contributes to the field of natural language processing (NLP) and its application in the labor market, particularly in non-English contexts.

Computation and Language
Enhanced Algorithmic Job Matching based on a Comprehensive Candidate Profile using NLP and Machine Learning

Vishnu S Pendyala,Saumya Goyal,Nishtha Atrey,Tina Aggarwal

DOI: https://doi.org/10.1109/BigDataService55688.2022.00040

2022-08-01

Abstract:We propose to aid the hiring process by automatic profiling of a candidate’s social media presence and using it in conjunction with their resume and other information to suggest an employability score and an emotional intelligence indicator. Algorithmic hiring usually supports the hiring company and does not factor in social media presence of the candidate. The work we propose in this paper can also be used by the job seekers to evaluate their chances and uses a comprehensive profile of the candidate. Our approach uses social media profiling using APIs and web crawlers, evaluating soft skills, shortlisting candidates based on keywords, skill-set, and educational requirements and suggesting a match using Machine Learning and NLP techniques. The resulting application provides a faster, accurate, efficient, and relatively bias-free recruitment process to the companies and to the job seekers.

Computer Science,Business
Research on Data Analysis and Visualization of Recruitment Positions Based on Text Mining

Yuanyuan Chen,Ruijie Pan

DOI: https://doi.org/10.1155/2022/9047202

2022-07-21

Advances in Multimedia

Abstract:With the rapid development of the Internet and the impact of COVID-19, online recruitment has gradually become the mainstream form of recruitment. However, existing online recruitment platforms fail to fully combine the job seekers' demands for salary, region, benefits, and other aspects, which cloud not display the information related to recruitment positions in a multidimensional way. To solve this problem, this paper firstly uses a web crawler to collect job information from recruitment websites based on keywords retrieved by users, then extracts job information using regular expressions, and cleans and processes the extracted job information using third-party libraries such as Pandas and NumPy. Finally, through the probabilistic theme model of text mining, the topic model of job description content in the recruitment information is modeled. Combining with the django development framework and related visualization technology, the relationship among education requirement, experience requirement, job location, salary, and other aspects in the recruitment information is visually displayed in a multidimensional way. At the same time, the GM model is used to realize the gray prediction of the number of employment personnel in related industries, which provides employment reference for the majority of job seekers and enterprises.
JobRecoGPT -- Explainable job recommendations using LLMs

Preetam Ghosh,Vaishali Sadaphal

2023-09-21

Abstract:In today's rapidly evolving job market, finding the right opportunity can be a daunting challenge. With advancements in the field of AI, computers can now recommend suitable jobs to candidates. However, the task of recommending jobs is not same as recommending movies to viewers. Apart from must-have criteria, like skills and experience, there are many subtle aspects to a job which can decide if it is a good fit or not for a given candidate. Traditional approaches can capture the quantifiable aspects of jobs and candidates, but a substantial portion of the data that is present in unstructured form in the job descriptions and resumes is lost in the process of conversion to structured format. As of late, Large Language Models (LLMs) have taken over the AI field by storm with extraordinary performance in fields where text-based data is available. Inspired by the superior performance of LLMs, we leverage their capability to understand natural language for capturing the information that was previously getting lost during the conversion of unstructured data to structured form. To this end, we compare performance of four different approaches for job recommendations namely, (i) Content based deterministic, (ii) LLM guided, (iii) LLM unguided, and (iv) Hybrid. In this study, we present advantages and limitations of each method and evaluate their performance in terms of time requirements.

Artificial Intelligence

Computational Job Market Analysis with Natural Language Processing

Deep Learning-based Computational Job Market Analysis: A Survey on Skill Extraction and Classification from Job Postings

A Big Data Analysis of Job Position Status Based on Natural Language Processing

Development and Evaluation of Task-Specific NLP Framework in China.

Understanding and Modeling Job Marketplace with Pretrained Language Models

Identifying Labor Market Competitors with Machine Learning Based on Maimai Platform

Occupational models from 42 million unstructured job postings

Resume Parser Analysis Using Machine Learning and Natural Language Processing

Data-driven HR - Résumé Analysis Based on Natural Language Processing and Machine Learning

NNOSE: Nearest Neighbor Occupational Skill Extraction

Skill Extraction from Job Postings using Weak Supervision

Resume Analysis Using Machine Learning and Natural Language Processing

Machine Learned Resume-Job Matching Solution

ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain

Automated staff assignment for building maintenance using natural language processing

RESUME SUMMARIZER AND JOB DESCRIPTION MATCHER USING NATURAL LANGUAGE PROCESSING AND SPACY

The Future of ChatGPT-enabled Labor Market: A Preliminary Study in China

Comparative Analysis of Encoder-Based NER and Large Language Models for Skill Extraction from Russian Job Vacancies

Enhanced Algorithmic Job Matching based on a Comprehensive Candidate Profile using NLP and Machine Learning

Research on Data Analysis and Visualization of Recruitment Positions Based on Text Mining

JobRecoGPT -- Explainable job recommendations using LLMs