Metrics for assessing similarity of jobs

Leaetta M. Hough,Teresa L. Russell

DOI: https://doi.org/10.1017/iop.2021.129

2022-03-01

Industrial and Organizational Psychology

Abstract://static.cambridge.org/content/id/urn%3Acambridge.org%3Aid%3Aarticle%3AS1754942621001292/resource/name/firstPage-S1754942621001292a.jpg

What problem does this paper attempt to address?

A Metric Learning Based Approach to Evaluate Task-Specific Time Series Similarity

Yang Lu,Wayne Xin Zhao,Hongfei Yan,Xiaoming Li

DOI: https://doi.org/10.1007/978-3-642-38562-9_32

2013-01-01

Abstract:A variety of methods have been proposed to measure time series similarity, such as Dynamic Time Warping and Edit distance. Although these methods have been shown to be effective and useful in various data mining tasks, they seldom consider task-specific information. Without consideration of task-specific information, the similarity measures may not work quite well on specific tasks. In this paper, we investigate how to learn task-specific time series similarity measures. We adopt metric learning as the principled approach, and we proposed two novel models based on metric learning to evaluate task-specified time series similarity. We construct our test collection based on real data from Renren Games data. Extensive experimental results show that our proposed methods are very effective.
Measurement of Similarity in Academic Contexts

Omid Mahian,Marius Treutwein,Patrice Estelle,Somchai Wongwises,Dongsheng Wen,Giulio Lorenzini,Ahmet Selim Dalkilic,Wei-Mon Yan,Ahmet Z. Sahin

DOI: https://doi.org/10.3390/publications5030018

2017-01-01

Publications

Abstract:We propose some reflections, comments and suggestions about the measurement of similar and matched content in scientific papers and documents, and the need to develop appropriate tools and standards for an ethically fair and equitable treatment of authors.
Occupation similarity through bipartite graphs

Pavle Boškoski,Matija Perne,Tjaša Redek,Biljana Mileva Boshkoska

DOI: https://doi.org/10.48550/arXiv.2202.11064

2022-02-17

Abstract:Similarity between occupations is a crucial piece of information when making career decisions. However, the notion of a single and unified occupation similarity measure is more of a limitation than an asset. The goal of the study is to assess multiple explainable occupation similarity measures that can provide different insights into inter-occupation relations. Several such measures are derived using the framework of bipartite graphs. Their viability is assessed on more than 450,000 job transitions occurring in Slovenia in the period between 2012 and 2021. The results support the hypothesis that several similarity measures are plausible and that they present different feasible career paths. The complete implementation and part of the datasets are available at <a class="link-external link-https" href="https://repo.ijs.si/pboskoski/bipartite_job_similarity_code" rel="external noopener nofollow">this https URL</a>.

Social and Information Networks,Artificial Intelligence
Measuring the similarity of charts in graphical statistics

Krzysztof Górnisiewicz,Zbigniew Palka,Waldemar Ratajczak

DOI: https://doi.org/10.1038/s41598-024-56156-5

IF: 4.6

2024-03-24

Scientific Reports

Abstract:Figures used in statistics and other sciences play a vital role in understanding and analyzing the problems under study. Due to the complexity and diversity of these problems, figures such as cartograms, choropleth maps, or radar charts take various geometric forms. Their visual evaluation from the view of geometric similarity is essential but insufficient. This paper proposes and theoretically justifies new metrics based on graph theory. They make it possible to quickly determine the degree of similarity of the statistical figures used in the research procedure. The new metrics were used to 1. Determine the similarity of the domestic route networks of major U.S. airlines, 2. Determine the similarity of the distribution of votes cast in U.S. presidential election in each state in 2016 and 2020, 3. Compare radar charts of some countries, constructed based on the Global Competitiveness Index, 4. Analyze the similarity of neutrosophic double line graphs representing sets of approximate (neutrosophic) numbers. This improves analytical capabilities concerning various processes mapped with well-known types of statistical charts, such as choropleth maps, radar charts, etc.

multidisciplinary sciences
Calculating Similarity of Javadoc Comments

D. V. Koznov,E. Yu. Ledeneva,D. V. Luciv,P. I. Braslavski

DOI: https://doi.org/10.1134/s0361768824010043

2024-05-23

Programming and Computer Software

Abstract:Code comments are an essential part of software documentation. Many software projects suffer from the problem of low-quality comments that are often produced by copy-paste. In case of similar methods, classes, etc. copy-pasted comments with minor modifications are justified. However, in many cases this approach leads to degraded documentation quality and, subsequently, to problematic maintenance and development of the project. In this study, we address the problem of near-duplicate code comments detection, which can potentially improve software documentation. We have conducted a thorough evaluation of traditional string similarity metrics and modern machine learning methods. In our experiment, we use a collection of Javadoc comments from four industrial open-source Java projects. We have found out that LCS (Longest Common Subsequence) is the best similarity algorithm taking into account both quality (Precision 94%, Recall 74%) and performance.

computer science, software engineering
A Semantic-Rich Similarity Measure in Heterogeneous Information Networks.

Yu Zhou,Jianbin Huang,He Li,Heli Sun,Yan Peng,Yueshen Xu

DOI: https://doi.org/10.1016/j.knosys.2018.05.010

IF: 8.139

2018-01-01

Knowledge-Based Systems

Abstract:Most of the existing similarity metrics in heterogeneous information networks depend on the pre-specified meta-path or meta-structure. This dependency may cause them to be sensitive to different meta-paths or meta-structures. In this paper, we propose a stratified meta-structure-based similarity measure named SMSS in heterogeneous information networks. The stratified meta-structure can be constructed automatically and capture rich semantics.Then, we define the commuting matrix of the stratified meta-structure by virtue of the commuting matrices of meta-paths and meta-structures. As a result, the SMSS is defined by virtue of this commuting matrix. Experimental evaluations show that the existing metrics are sensitive to different meta-paths or meta-structures and that the proposed SMSS outperforms the state-of-the-art metrics in terms of ranking and clustering.
An Examination of Similarity between a Costume Designer and a Production Designer: Job Analysis Goes to Hollywood

Steven E. Williams,Jon M. Werner

DOI: https://doi.org/10.1177/08863687221077729

2022-03-25

Abstract:This study examines the similarity of two unique positions in Hollywood—a costume designer and a production designer. It applies traditional job analysis and job evaluation methods in an industry with little history of doing so. Job data from interviews, questionnaires, and secondary sources were content analyzed for themes and categories. Quantitative data were collected and analyzed for supplemental evidence in comparing the similarity of the two positions. The study incorporated criteria from the California Fair Pay Act of 2016 in the assessment. Eight subject matter experts (SMEs) provided both narrative and quantitative data in response to the comprehensive job analysis. The results of the study show that these positions are sufficiently similar according to the criteria set forth in the California Fair Pay Act, raising issues concerning observed pay disparities between these positions. The findings also support the use of traditional methods of human resource management and industrial and organizational psychology in novel industries with unique positions. Finally, conclusions, limitations and implications are considered.
A Measure of the System Dependence of Automated Metrics

Pius von Däniken,Jan Deriu,Mark Cieliebak

2024-12-04

Abstract:Automated metrics for Machine Translation have made significant progress, with the goal of replacing expensive and time-consuming human evaluations. These metrics are typically assessed by their correlation with human judgments, which captures the monotonic relationship between human and metric scores. However, we argue that it is equally important to ensure that metrics treat all systems fairly and consistently. In this paper, we introduce a method to evaluate this aspect.

Computation and Language,Artificial Intelligence
Company Name Matching Using Job Market Data Enrichment

Andrei A. Ternikov

DOI: https://doi.org/10.1109/mitp.2024.3371179

2024-05-03

IT Professional

Abstract:This article contributes to the field of matching techniques by introducing a new algorithm based on labor market data enrichment. This approach is able to collect and balance the training and test samples for data integration purposes. By setting thresholds for textual matching and geographic proximity, it simplifies the process of finding suitable company matches. Based on insufficiently studied datasets, the experimental findings show that the performance evaluation of proposed models differs depending on the similarity thresholds used.

computer science, information systems,telecommunications, software engineering
HN-Sim: A Structural Similarity Measure over Object-Behavior Networks.

Jiazhen Nian,Shanshan Wang,Yan Zhang

DOI: https://doi.org/10.1007/978-3-642-53914-5_5

2013-01-01

Abstract:Measurement of similarity is a critical work for many applications such as text analysis, link prediction and recommendation. However, existing work stresses on content and rarely involves structural features. Even fewer methods are applicable for heterogeneous network, which is prevalent in the real world, such as bibliographic information network. To address this problem, we propose a new measurement of similarity from the perspective of the heterogeneous structure. Heterogeneous neighborhood is utilized to instantiate the topological features and categorize the related nodes in graph model. We make a comparison between our measurement and some traditional ones with the real data in DBLP1 and Flickr2. Manual evaluation shows that our method outperforms the traditional ones. © Springer-Verlag 2013.
Description and Evaluation of Semantic Similarity Measures Approaches

Thabet Slimani

DOI: https://doi.org/10.5120/13897-1851

2013-10-30

Abstract:In recent years, semantic similarity measure has a great interest in Semantic Web and Natural Language Processing (NLP). Several similarity measures have been developed, being given the existence of a structured knowledge representation offered by ontologies and corpus which enable semantic interpretation of terms. Semantic similarity measures compute the similarity between concepts/terms included in knowledge sources in order to perform estimations. This paper discusses the existing semantic similarity methods based on structure, information content and feature approaches. Additionally, we present a critical evaluation of several categories of semantic similarity approaches based on two standard benchmarks. The aim of this paper is to give an efficient evaluation of all these measures which help researcher and practitioners to select the measure that best fit for their requirements.

Computation and Language
Evaluation metrics and statistical tests for machine learning

Oona Rainio,Jarmo Teuho,Riku Klén

DOI: https://doi.org/10.1038/s41598-024-56706-x

IF: 4.6

2024-03-14

Scientific Reports

Abstract:Research on different machine learning (ML) has become incredibly popular during the past few decades. However, for some researchers not familiar with statistics, it might be difficult to understand how to evaluate the performance of ML models and compare them with each other. Here, we introduce the most common evaluation metrics used for the typical supervised ML tasks including binary, multi-class, and multi-label classification, regression, image segmentation, object detection, and information retrieval. We explain how to choose a suitable statistical test for comparing models, how to obtain enough values of the metric for testing, and how to perform the test and interpret its results. We also present a few practical examples about comparing convolutional neural networks used to classify X-rays with different lung infections and detect cancer tumors in positron emission tomography images.

multidisciplinary sciences
Metric Similarity Joins Using MapReduce

ChenGang,YangKeyu,ChenLu,GaoYunjun,ZhengBaihua,ChenChun

IF: 9.235

2017-01-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:Given two object sets Q and O, a metric similarity join finds similar object pairs according to a certain criterion. This operation has a wide variety of applications in data cleaning and data mini...
A Semantic-Rich Similarity in Heterogeneous Information Networks

Yu Zhou,Jianbin Huang,Heli Sun,Yizhou Sun,Hong Cheng

2018-01-01

Abstract:Measuring the similarities between objects in information networks has fundamental importance in recommendation systems, clustering and web search. The existing metrics depend on the meta path or meta structure specified by users. In this paper, we propose a stratified meta structure based similarity SMSS in heterogeneous information networks. The stratified meta structure can be constructed automatically and capture rich semantics. Then, we define the commuting matrix of the stratified meta structure by virtue of the commuting matrices of meta paths and meta structures. As a result, SMSS is defined by virtue of these commuting matrices. Experimental evaluations show that the proposed SMSS on the whole outperforms the state-of-the-art metrics in terms
Non-Metric Similarity Measures

Kai M. Ting

DOI: https://doi.org/10.21236/ada622081

2015-01-01

Abstract:: This was an extension of mass theory to non-metric similarity measures. The non-metric similarity measures were created as a generalization of mass estimation from a unary function to a binary function. Unlike the traditional similarity measure that is based on geometric difference between two instances, the mass-based measure takes data distribution into account. It is demonstrated that the new measure results in better performance in applying to information retrieval task. A derivative of mass measure called relative mass was also investigated using three implementations. The research in relative mass was expanded to two tasks: In anomaly detection relative mass is used to overcome one weakness of current mass-based anomaly detectors using a tree-based approach and a nearest-neighbor-based approach; in clustering, relative mass is used to recondition density-based clustering algorithms to successfully find clusters with varying densities.
Ranking occupations by their proximity to workers' profiles

Mirjam Bächli,Hélène Benghalem,Doriana Tinello,Damaris Aschwanden,Sascha Zuber,Matthias Kliegel,Michele Pellizzari,Rafael Lalive

DOI: https://doi.org/10.1186/s41937-024-00125-2

Abstract:Information friction makes it difficult for job seekers to find new employment opportunities. We propose a method for providing individual-specific occupation recommendations by ranking occupations based on their proximity to the worker's profile. We identify a set of twelve skills, abilities and work styles that capture the worker-oriented requirements of all occupations and discuss how to measure these items using online questions and tasks. We use the Euclidean distance between the measured items pertaining to a worker and the requirements of an occupation to measure the proximity between job seekers and occupations. We show that the proximity between job seekers' profiles and their preunemployment occupation predicts their intention to change occupations, thus suggesting that our method captures a meaningful conceptualization of mismatch. We also show that our method generates recommendations that differ from the previous occupations of mismatched job seekers, thereby potentially expanding their search scope.
Measuring Similarity Based on Link Information: A Comparative Study

Hongyan Liu,Jun He,Dan Zhu,Charles X. Ling,Xiaoyong Du

DOI: https://doi.org/10.1109/tkde.2012.194

IF: 9.235

2013-01-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:Measuring similarity between objects is a fundamental task in domains such as data mining, information retrieval, and so on. Link-based similarity measures have attracted the attention of many researchers and have been widely applied in recent years. However, most previous works mainly focus on introducing new link-based measures, and seldom provide theoretical as well as experimental comparisons with other measures. Thus, selecting the suitable measure in different situations and applications is difficult. In this paper, a comprehensive analysis and critical comparison of various link-based similarity measures and algorithms are presented. Their strengths and weaknesses are discussed. Their actual runtime performances are also compared via experiments on benchmark data sets. Some novel and useful guidelines for users to choose the appropriate link-based measure for their applications are discovered.
Measuring data similarity in population-based structural health monitoring using distance metrics

Chandula T Wickramarachchi,Eoghan Maguire,Elizabeth J Cross,Keith Worden

DOI: https://doi.org/10.1177/14759217231207526

2023-11-27

Structural Health Monitoring

Abstract:Structural Health Monitoring, Ahead of Print. Population-based structural health monitoring (PBSHM) expands structural health monitoring (SHM) from a single structure to a group of structures, allowing inferences to be made within and between populations by transferring knowledge across them. Within the populations of interest, the similarity of structures, via their corresponding data, should be assessed to successfully implement PBSHM. This paper focusses on using distance metrics to assess similarity at the very start of the analysis chain, to discover information about a population for which there is little prior knowledge and before any analysis has taken place on individual structures. By doing so, it is possible to quickly and automatically identify abnormalities within the population, group similarly behaving structures together, and inform further decisions. The suitability of several candidate metrics that are not widely employed in SHM are tested using a number of commonly occurring feature behaviours, such as varying amplitudes and temporary mean shifts. The effect of data normalisation/standardisation on the metrics is also explored to identify interesting behaviours within the data. A case study is then presented where distance metrics are used to discover similarities and dissimilarities within temperature data from turbines in an offshore wind farm.

engineering, multidisciplinary,instruments & instrumentation
SIMpat: A Synthetic Benchmark for Similarity Metrics on Patient Representations

Jean-Virgile Voegeli,Mina Bjelogrlic,Christophe Gaudet-Blavignac,Richard Dubos,Myriam Zimmermann,Adel Bensahla Talet,Yuanyuan Zheng,Julien Ehrsam,Christian Lovis

DOI: https://doi.org/10.3233/SHTI240739

2024-08-22

Abstract:Similarity and clustering tasks based on data extracted from electronic health records on the patient level suffer from the curse of dimensionality and the lack of inter-patient data comparability. Indeed, for many health institutions, there are many more variables, and ways of expressing those variables to represent patients than patients sharing the same set of data. To lower redundancy and increase interoperability one strategy is to map data to semantic-driven representations through medical knowledge graphs such as SNOMED-CT. However, patient similarity metrics based on this knowledge-graph information lack quantitative evaluation and comparisons with pure data-driven methods. The reasons are twofold, firstly, it is hard to conceptually assess and formalize a gold-standard similarity between patients resulting in poor inter-annotator agreement in qualitative evaluations. Secondly, the community has been lacking a clear benchmark to compare existing metrics developed by scientific communities coming from various fields such as ontology, data science, and medical informatics. This study proposes to leverage the known challenges of evaluating patient similarities by proposing SIMpat, a synthetic benchmark to quantitatively evaluate available metrics, based on controlled cohorts, which could later be used to assess their sensibility regarding aspects such as the sparsity of variables or specificities of patient disease patterns.
Machine Learning Evaluation Metric Discrepancies across Programming Languages and Their Components: Need for Standardization

Mohammad R. Salmanpour,Morteza Alizadeh,Ghazal Mousavi,Saba Sadeghi,Sajad Amiri,Mehrdad Oveisi,Arman Rahmim,Ilker Hacihaliloglu

2024-11-19

Abstract:This study evaluates metrics for tasks such as classification, regression, clustering, correlation analysis, statistical tests, segmentation, and image-to-image (I2I) translation. Metrics were compared across Python libraries, R packages, and Matlab functions to assess their consistency and highlight discrepancies. The findings underscore the need for a unified roadmap to standardize metrics, ensuring reliable and reproducible ML evaluations across platforms. This study examined a wide range of evaluation metrics across various tasks and found only some to be consistent across platforms, such as (i) Accuracy, Balanced Accuracy, Cohens Kappa, F-beta Score, MCC, Geometric Mean, AUC, and Log Loss in binary classification; (ii) Accuracy, Cohens Kappa, and F-beta Score in multi-class classification; (iii) MAE, MSE, RMSE, MAPE, Explained Variance, Median AE, MSLE, and Huber in regression; (iv) Davies-Bouldin Index and Calinski-Harabasz Index in clustering; (v) Pearson, Spearman, Kendall's Tau, Mutual Information, Distance Correlation, Percbend, Shepherd, and Partial Correlation in correlation analysis; (vi) Paired t-test, Chi-Square Test, ANOVA, Kruskal-Wallis Test, Shapiro-Wilk Test, Welchs t-test, and Bartlett's test in statistical tests; (vii) Accuracy, Precision, and Recall in 2D segmentation; (viii) Accuracy in 3D segmentation; (ix) MAE, MSE, RMSE, and R-Squared in 2D-I2I translation; and (x) MAE, MSE, and RMSE in 3D-I2I translation. Given observation of discrepancies in a number of metrics (e.g. precision, recall and F1 score in binary classification, WCSS in clustering, multiple statistical tests, and IoU in segmentation, amongst multiple metrics), this study concludes that ML evaluation metrics require standardization and recommends that future research use consistent metrics for different tasks to effectively compare ML techniques and solutions.

Machine Learning,Software Engineering,Computational Physics

Metrics for assessing similarity of jobs

A Metric Learning Based Approach to Evaluate Task-Specific Time Series Similarity

Measurement of Similarity in Academic Contexts

Occupation similarity through bipartite graphs

Measuring the similarity of charts in graphical statistics

Calculating Similarity of Javadoc Comments

A Semantic-Rich Similarity Measure in Heterogeneous Information Networks.

An Examination of Similarity between a Costume Designer and a Production Designer: Job Analysis Goes to Hollywood

A Measure of the System Dependence of Automated Metrics

Company Name Matching Using Job Market Data Enrichment

HN-Sim: A Structural Similarity Measure over Object-Behavior Networks.

Description and Evaluation of Semantic Similarity Measures Approaches

Evaluation metrics and statistical tests for machine learning

Metric Similarity Joins Using MapReduce

A Semantic-Rich Similarity in Heterogeneous Information Networks

Non-Metric Similarity Measures

Ranking occupations by their proximity to workers' profiles

Measuring Similarity Based on Link Information: A Comparative Study

Measuring data similarity in population-based structural health monitoring using distance metrics

SIMpat: A Synthetic Benchmark for Similarity Metrics on Patient Representations

Machine Learning Evaluation Metric Discrepancies across Programming Languages and Their Components: Need for Standardization