Abstract:This work tackles the problem of fuzzy joining of strings that naturally tokenize into meaningful substrings, e.g., full names. Tokenized-string joins have several established applications in the context of data integration and cleaning. This work is primarily motivated by fraud detection, where attackers slightly modify tokenized strings, e.g., names on accounts, to create numerous identities that she can use to defraud service providers, e.g., Google, and LinkedIn. To detect such attacks, all the accounts are pair-wise compared, and the resulting similar accounts are considered suspicious and are further investigated. Comparing the tokenized-string features of a large number of accounts requires an intuitive tokenized-string distance that can detect subtle edits introduced by an adversary, and a very scalable algorithm. This is not achievable by existing distance measure that are unintuitive, hard to tune, and whose join algorithms are serial and hence unscalable. We define a novel intuitive distance measure between tokenized strings, Normalized Setwise Levenshtein Distance (NSLD). To the best of our knowledge, NSLD is the first metric proposed for comparing tokenized strings. We propose a scalable distributed framework, Tokenized-String Joiner (TSJ), that adopts existing scalable string-join algorithms as building blocks to perform NSLD-joins. We carefully engineer optimizations and approximations that dramatically improve the efficiency of TSJ. The effectiveness of the TSJ framework is evident from the evaluation conducted on tens of millions of tokenized-string names from Google accounts. The superiority of the tokenized-string-specific TSJ framework over the general-purpose metric-spaces joining algorithms has been established.

Scalable Metric Similarity Join Using MapReduce

An Efficient MapReduce Algorithm for Similarity Join in Metric Spaces

Metric Similarity Joins Using MapReduce

Improving Distributed Similarity Join in Metric Space with Error-bounded Sampling

Metric Similarity Joins Using Mapreduce (Extended Abstract)

Join Query Optimization Based on MapReduce under Skewed Data

Improve Semantic Web Services Discovery Through Similarity Search in Metric Space

Performance Evaluation for Distributed Join Based on MapReduce.

Massjoin: A Mapreduce-Based Method for Scalable String Similarity Joins

MELODY-JOIN: Efficient Earth Mover's Distance Similarity Joins Using MapReduce

Heads-Join: Efficient Earth Mover's Distance Similarity Joins on Hadoop

An Effective High-Performance Multiway Spatial Join Algorithm with Spark

Efficient String Similarity Join in Multi-Core and Distributed Systems.

MSQL: Efficient Similarity Search in Metric Spaces Using SQL.

A New Design of High-Performance Large-Scale GIS Computing at a Finer Spatial Granularity: A Case Study of Spatial Join with Spark for Sustainability

Scalable Parallel Join for Huge Tables

Scalable Similarity Joins of Tokenized Strings

DIMS: Distributed Index for Similarity Search in Metric Spaces

SiMPSON: Efficient Similarity Search in Metric Spaces over P2P Structured Overlay Networks

Distributed Similarity Queries in Metric Spaces