Abstract:Motivation: Many tasks in sequence analysis ask to identify biologically related sequences in a large set. The edit distance, being a sensible model for both evolution and sequencing error, is widely used in these tasks as a measure. The resulting computational problem-to recognize all pairs of sequences within a small edit distance-turns out to be exceedingly difficult, since the edit distance is known to be notoriously expensive to compute and that all-versus-all comparison is simply not acceptable with millions or billions of sequences. Among many attempts, we recently proposed the locality-sensitive bucketing (LSB) functions to meet this challenge. Formally, a (d1,d2)-LSB function sends sequences into multiple buckets with the guarantee that pairs of sequences of edit distance at most d1 can be found within a same bucket while those of edit distance at least d2 do not share any. LSB functions generalize the locality-sensitive hashing (LSH) functions and admit favorable properties, with a notable highlight being that optimal LSB functions for certain (d1,d2) exist. LSB functions hold the potential of solving above problems optimally, but the existence of LSB functions for more general (d1,d2) remains unclear, let alone constructing them for practical use. Results: In this work, we aim to utilize machine learning techniques to train LSB functions. With the development of a novel loss function and insights in the neural network structures that can potentially extend beyond this specific task, we obtained LSB functions that exhibit nearly perfect accuracy for certain (d1,d2), matching our theoretical results, and high accuracy for many others. Comparing to the state-of-the-art LSH method Order Min Hash, the trained LSB functions achieve a 2- to 5-fold improvement on the sensitivity of recognizing similar sequences. An experiment on analyzing erroneous cell barcode data is also included to demonstrate the application of the trained LSB functions. Availability and implementation: The code for the training process and the structure of trained models are freely available at https://github.com/Shao-Group/lsb-learn.

Grasp For Low Autocorrelated Binary Sequences

Dual-Step Optimization for Binary Sequences with High Merit Factors

Learning directed acyclic graphs based on sparsest permutations

Binary Sequences With Good Aperiodic Autocorrelations Using Cross-Entropy Method

Efficient Heuristic Method of Search for Binary Sequences with Good Aperiodic Autocorrelations

Efficient Generation of Low Autocorrelation Binary Sequences

A Learning Search Algorithm for the Restricted Longest Common Subsequence Problem

Solving unconstrained binary polynomial programs with limited reach: Application to low autocorrelation binary sequences

Computational Searching of Long Skew-symmetric Binary Sequences with High Merit Factors

An Effective Grasp and Tabu Search for the 0-1 Quadratic Knapsack Problem

AdaLead: A simple and robust adaptive greedy search algorithm for sequence design

An Efficient Heuristics Search For Binary Sequences With Good Aperiodic Autocorrelations

Learning locality-sensitive bucketing functions

Parallel self-avoiding walks for a low-autocorrelation binary sequences problem

Towards easier and faster sequence labeling for natural language processing: A search-based probabilistic online learning framework (SAPO)

Phase Code Discovery for Pulse Compression Radar: A Genetic Algorithm Approach

A simple and effective hybrid genetic search for the job sequencing and tool switching problem

RANDOM APPROXIMATED GREEDY SEARCH FOR FEATURE SUBSET SELECTION

Recognizing Short Coding Sequences of Prokaryotic Genome Using a Novel Iteratively Adaptive Sparse Partial Least Squares Algorithm.

On A Class of Greedy Sparse Recovery Algorithms -- A High Dimensional Approach

Finding automatic sequences with few correlations