Abstract:Entity resolution (ER) is a key data integration problem. Despite the efforts in 70+ years in all aspects of ER, there is still a high demand for democratizing ER - humans are heavily involved in labeling data, performing feature engineering, tuning parameters, and defining blocking functions. With the recent advances in deep learning, in particular distributed representation of words (a.k.a. word embeddings), we present a novel ER system, called DeepER, that achieves good accuracy, high efficiency, as well as ease-of-use (i.e., much less human efforts). For accuracy, we use sophisticated composition methods, namely uni- and bi-directional recurrent neural networks (RNNs) with long short term memory (LSTM) hidden units, to convert each tuple to a distributed representation (i.e., a vector), which can in turn be used to effectively capture similarities between tuples. We consider both the case where pre-trained word embeddings are available as well the case where they are not; we present ways to learn and tune the distributed representations. For efficiency, we propose a locality sensitive hashing (LSH) based blocking approach that uses distributed representations of tuples; it takes all attributes of a tuple into consideration and produces much smaller blocks, compared with traditional methods that consider only a few attributes. For ease-of-use, DeepER requires much less human labeled data and does not need feature engineering, compared with traditional machine learning based approaches which require handcrafted features, and similarity functions along with their associated thresholds. We evaluate our algorithms on multiple datasets (including benchmarks, biomedical data, as well as multi-lingual data) and the extensive experimental results show that DeepER outperforms existing solutions.

Enhancing Entity Resolution with a hybrid Active Machine Learning framework: Strategies for optimal learning in sparse datasets

ERABQS: entity resolution based on active machine learning and balancing query strategy

Low-resource Deep Entity Resolution with Transfer and Active Learning

Low-resource entity resolution with domain generalization and active learning

DeepER -- Deep Entity Resolution

Active Deep Learning on Entity Resolution by Risk Sampling

Gradual Machine Learning for Entity Resolution

On Leveraging Large Language Models for Enhancing Entity Resolution: A Cost-efficient Approach

CollaborEM: A Self-supervised Entity Matching Framework Using Multi-features Collaboration

CollaborER: A Self-supervised Entity Resolution Framework Using Multi-features Collaboration

Enabling Quality Control for Entity Resolution: A Human and Machine Cooperation Framework

Entity Matching by Pool-Based Active Learning

BoostER: Leveraging Large Language Models for Enhancing Entity Resolution

Efficient Entity Resolution on Heterogeneous Records

Cost-Effective In-Context Learning for Entity Resolution: A Design Space Exploration

An in-depth analysis of pre-trained embeddings for entity resolution

A Comprehensive Benchmark Framework for Active Learning Methods in Entity Matching

Synthesizing Privacy Preserving Entity Resolution Datasets

End-to-End Multi-Perspective Matching for Entity Resolution

Deep Sequence-to-Sequence Entity Matching for Heterogeneous Entity Resolution

Domain Adaptation for Deep Entity Resolution