Multi-selection and Distributed Sorting

Xiaofang Zhou,Zhiquan Jin

1987-01-01

Abstract:The definition of multiselection is given, as are two related algorithms, for sequential and distributed Quicksort-based (m,n)-selection. A distributed sorting algorithm based on distributed multiselection, which improves on existing algorithms, is also proposed.

What problem does this paper attempt to address?

Distributed Privacy-Aware Fast Selection Algorithm for Large-Scale Data.

Hao Liu,Jiming Chen

DOI: https://doi.org/10.1109/tpds.2017.2761344

IF: 5.3

2018-01-01

IEEE Transactions on Parallel and Distributed Systems

Abstract:Finding the k smallest/largest element of a large array, i.e., k-selection is a fundamental supporting algorithm in data analysis. Due to the fact that big data born in geo-distributed environments, it especially requires communication-efficient distributed k-selection, besides typical computation and memory efficiency. Moreover, sensitive organizations make data privacy a rigorous precondition for their participation in such distributed statistical analysis for common profit. To this end, we propose a Distributed Privacy-Aware Median (DPAM) selection algorithm for median selection in distributed large-scale data while preserving local statistics privacy, and extend it to arbitrary k-selection. DPAM utilizes mean to approximate median, via contraction of the standard deviation. It is the theoretical fastest with a worst computation complexity of O(N), and also highly efficient in communication overhead (in logarithm of data range). To preserve is an element of-differential privacy of local statistics, DPAM randomly adds dummy elements (the number follows a rounded Laplacian distribution) to local data. The noise does not degrade the estimation precision or convergence rate. Performance of DPAM is compared with centralized/distributed quick select and optimization, in terms of complexity and privacy preserving ability. Extensive simulation and experiment results show the higher efficiency of DPAM.
Synergistic Sorting, MultiSelection and Deferred Data Structures on MultiSets

Jérémy Barbay,Carlos Ochoa,Srinivasa Rao Satti

DOI: https://doi.org/10.48550/arXiv.1608.06666

2016-10-01

Abstract:Karp et al. (1988) described Deferred Data Structures for Multisets as "lazy" data structures which partially sort data to support online rank and select queries, with the minimum amount of work in the worst case over instances of size $n$ and number of queries $q$ fixed (i.e., the query size). Barbay et al. (2016) refined this approach to take advantage of the gaps between the positions hit by the queries (i.e., the structure in the queries). We develop new techniques in order to further refine this approach and to take advantage all at once of the structure (i.e., the multiplicities of the elements), the local order (i.e., the number and sizes of runs) and the global order (i.e., the number and positions of existing pivots) in the input; and of the structure and order in the sequence of queries. Our main result is a synergistic deferred data structure which performs much better on large classes of instances, while performing always asymptotically as good as previous solutions. As intermediate results, we describe two new synergistic sorting algorithms, which take advantage of the structure and order (local and global) in the input, improving upon previous results which take advantage only of the structure (Munro and Spira 1979) or of the local order (Takaoka 1997) in the input; and one new multiselection algorithm which takes advantage of not only the order and structure in the input, but also of the structure in the queries. We described two compressed data structures to represent a multiset taking advantage of both the local order and structure, while supporting the operators rank and select on the multiset.

Data Structures and Algorithms
Parallel External Selection Algorithm on Distributed Memory Systems

cheng zhong,guoliang chen,chao yan

DOI: https://doi.org/10.1109/ICAPP.2002.1173581

2002-01-01

Abstract:We study parallel solutions to the problem of weighted multiselection to select r elements on given weighted-ranks from a set S of n weighted elements, where an element is on weighted rank k if it is the smallest element such that the aggregated weight ...
A Partitioning Selection Algorithm on Multiprocessors

Chen Guoliang

DOI: https://doi.org/10.1007/bf02943349

IF: 1.871

1988-01-01

Journal of Computer Science and Technology

Abstract:The so-called (m, n) selection, problem is the problem of selecting them smallest (or largest) elements fromn given numbers (n>m). With the development of parallel computers, much attention has been paid to the design of efficient algorithms of (m, n) problem for these machines. The parallel selection algorithm has been successful on networks, but seldom studied on the multiprocessing systems. This paper, based on a partitioning approach, proposes a partitioning algorithm of selection on multiprocessors using Valiant’s merging and sorting schemes. By means of this algorithm, (m, n) selection problem can be completed in paralleln/2 processors in timeO (lognloglogm-log(n/m)loglog(n/m))1).
High Performance Parallel Sort for Shared and Distributed Memory MIMD

Thoria Alghamdi,Gita Alaghband

DOI: https://doi.org/10.48550/arXiv.2003.01216

2020-03-03

Abstract:We present four high performance hybrid sorting methods developed for various parallel platforms: shared memory multiprocessors, distributed multiprocessors, and clusters taking advantage of existence of both shared and distributed memory. Merge sort, known for its stability, is used to design several of our algorithms. We improve its parallel performance by combining it with Quicksort. We present two models designed for shared memory MIMD (OpenMP): (a) a non-recursive Merge sort and (b) a hybrid Quicksort and Merge sort. The third model presented is designed for distributed memory MIMD (MPI) using a hybrid Quicksort and Merge sort. Our fourth model is designed to take advantage of the shared memory within individual nodes of todays cluster systems, and to eliminate all internal data transfers between different nodes, Our model implements a one-step MSD-Radix to distribute data in ten packets (MPI) while parallel cores of each node use Quicksort to sort their data partitions sequentially then merge and sort them in parallel employing the OpenMp. The performances of all developed models outperform the baseline performance. Hybrid Quicksort and Merge sort outperformed Hybrid Memory Parallel Merge Sort using Hybrid MSD-Radix and Quicksort in Cluster Platforms when sorting small size data, but with larger data the speedup of Hybrid Memory Parallel Merge Sort Using Hybrid MSD-Radix and Quicksort in Cluster Platforms becomes bigger and it keeps improving. The speedup of Distributed Memory Parallel Hybrid Quicksort and Merge Sort is the best.

Distributed, Parallel, and Cluster Computing
Parallel Sorting by Approximate Splitting for Multi-core Processors

Wei Wu,Pin Wu,Weilie Shang,Youtong Fang

DOI: https://doi.org/10.1109/cso.2010.72

2010-01-01

Abstract:Many pivot-based sorting methods benefit from balanced partition of data into smaller chunks with better parallelism and locality. However, it is difficult to partition data with the same high efficiency. This paper presents an improved partition method, Parallel Sorting by Approximate Splitting, which is based on an extend pivots selecting algorithm. Compared with other algorithm, such as PSRS, its splitter method is more flexibility and efficiency. Experiments which implemented by OpenMP in multi-cores environment and the theoretical analysis show the method is an efficiency algorithm with very good scalability.
Distributed Semi-supervised Multi-label Classification with Quantized Communication

Zhen Xu,Yifan Zhai,Ying Liu

DOI: https://doi.org/10.1145/3383972.3384055

2020-01-01

Abstract:In this paper, the problem of distributed multi-label classification (MLC) with a small portion of labeled data is considered, and a quantized distributed semi-supervised multi-label learning (QdS(2)ML(2)) algorithm is proposed. In the proposed algorithm, to utilize the information of both labeled and unlabeled data, information theoretic measures modeled by the kernel logistic regression function are used to design the loss function. Besides, to exploit the high-order label correlation over a network, a common low-dimensional subspace shared by multiple labels is learned in a distributed manner using quantized communication. Simulations on two real datasets are performed and results show that the proposed algorithm is very effective in solving MLC.
Data-centric workloads with MPI_Sort

P. Zulian,S. Ben Bader,G. Fourestey,R. Krause,D. Rossinelli

DOI: https://doi.org/10.1016/j.jpdc.2023.104833

IF: 4.542

2024-05-01

Journal of Parallel and Distributed Computing

Abstract:Sorting is a fundamental task in computing and plays a central role in information technology. The advent of rack-scale and warehouse-size data processing shaped the architecture of data analysis platforms towards supercomputing. In turn, established techniques on supercomputers have become relevant to a wider range of application domains. This work is concerned with multi-way mergesort with exact splitting on distributed memory architectures. At its core, our approach leverages a novel and parallel algorithm for multi-way selection problems. Remarkably concise, the algorithm relies on MPI_Allgather and MPI_ReduceScatter_block, two collective communication schemes that find hardware support in most high-end networks. A software implementation of our approach is used to process the Terabyte-size Data Challenge 2 signal, released by the SKA radio telescopes organization. On the supercomputer considered herein, our approach outperforms the state of the art by up to 2.6X using 9,216 cores. Our implementation is released as a compact open source library compliant to the MPI programming model. By supporting the most popular elementary key types, and arbitrary fixed-size value types, the library can be straightforwardly integrated into third-party MPI-based software.

computer science, theory & methods
Anytime Sorting Algorithms (Extended Version)

Emma Caizergues,François Durand,Fabien Mathieu

2024-05-14

Abstract:This paper addresses the anytime sorting problem, aiming to develop algorithms providing tentative estimates of the sorted list at each execution step. Comparisons are treated as steps, and the Spearman's footrule metric evaluates estimation accuracy. We propose a general approach for making any sorting algorithm anytime and introduce two new algorithms: multizip sort and Corsort. Simulations showcase the superior performance of both algorithms compared to existing methods. Multizip sort keeps a low global complexity, while Corsort produces intermediate estimates surpassing previous algorithms.

Data Structures and Algorithms
Parallel Multi-Deque Partition Dual-Deque Merge sorting algorithm using OpenMP

Sirilak Ketchaya,Apisit Rattanatranurak

DOI: https://doi.org/10.1038/s41598-023-33583-4

2023-04-19

Abstract:Quicksort is an important algorithm that uses the divide and conquer concept, and it can be run to solve any problem. The performance of the algorithm can be improved by implementing this algorithm in parallel. In this paper, the parallel sorting algorithm named the Multi-Deque Partition Dual-Deque Merge Sorting algorithm (MPDMSort) is proposed and run on a shared memory system. This algorithm contains the Multi-Deque Partitioning phase, which is a block-based parallel partitioning algorithm, and the Dual-Deque Merging phase, which is a merging algorithm without compare-and-swap operations and sorts the small data with the sorting function of the standard template library. The OpenMP library, which is an application programming interface used to develop the parallel implementation of this algorithm, is implemented in MPDMSort. Two computers (one with an Intel Xeon Gold 6142 CPU and the other with an Intel Core i7-11700 CPU) running Ubuntu Linux are used in this experiment. The results show that MPDMSort is faster than parallel balanced quicksort and multiway merge sort on the large random distribution data. A speedup of 13.81[Formula: see text] and speedup per thread of 0.86 can be obtained. Thus, developers can use these parallel partitioning and merging algorithms to improve the performance of related algorithms.
Array Sort: an Adaptive Sorting Algorithm on Multi-Thread

Xin Huang,Zhijing Liu,Jinyang Li

DOI: https://doi.org/10.1049/joe.2018.5154

2019-01-01

The Journal of Engineering

Abstract:Sorting is the most fundamental operation in database system. There are many classical sorting algorithms and among them the most commonly-used sorting algorithm in modern database system is merge sort. Merge sort is an efficient, general-purpose, comparison-based sorting algorithm. As merge sort is based on a divide and conquer model, it has found wide use in parallel computing algorithms. In this study, the authors present an adaptive sorting algorithm based on merge sort which is called array sort. Array sort not only takes advantages of merge sort, but also simplify the merge step by using a tag array. The proposed implementation consists of three phases: tag array creation phase, tag array split phase and list merge phase. Instead of just evaluating array sort in a single thread environment, the authors also run their experiments on multi-thread to test how array sort performs.
Multiobjective Feature Selection for Microarray Data Via Distributed Parallel Algorithms.

Bin Cao,Jianwei Zhao,Po Yang,Peng Yang,Xin Liu,Jun Qi,Andrew Simpson,Mohamed Elhoseny,Irfan Mehmoode,Khan Muhammad

DOI: https://doi.org/10.1016/j.future.2019.02.030

IF: 7.307

2019-01-01

Future Generation Computer Systems

Abstract:Many real-world problems are large in scale and hence difficult to address. Due to the large number of features in microarray datasets, feature selection and classification are even more challenging for such datasets. Not all of these numerous features contribute to the classification task, and some even impede performance. Through feature selection, a feature subset that contains only a small quantity of essential features can be generated to increase the classification accuracy and significantly reduce the time consumption. In this paper, we construct a multiobjective feature selection model that simultaneously considers the classification error, the feature number and the feature redundancy. For this model, we propose several distributed parallel algorithms based on different encodings and an adaptive strategy. Additionally, to reduce the time consumption, various tactics are employed, including a feature number constraint, distributed parallelism and sample-wise parallelism. For a batch of microarray datasets, the proposed algorithms are superior to several state-of-the-art multiobjective evolutionary algorithms in terms of both effectiveness and efficiency.
Sorting and Selection in Posets

Constantinos Daskalakis,Richard M. Karp,Elchanan Mossel,Samantha J. Riesenfeld,Elad Verbin

DOI: https://doi.org/10.1137/070697720

2011-01-01

SIAM Journal on Computing

Abstract:Classical problems of sorting and searching assume an underlying linear ordering of the objects being compared. In this paper, we study these problems in the context of partially ordered sets, in which some pairs of objects are incomparable. This generalization is interesting from a combinatorial perspective, and it has immediate applications in ranking scenarios where there is no underlying linear ordering, e.g., conference submissions. It also has applications in reconstructing certain types of networks, including biological networks. Our results represent significant progress over previous results from two decades ago by Faigle and Turn. In particular, we present the first algorithm that sorts a width-w poset of size n with query complexity $O(n(w+\log n))$ and prove that this query complexity is asymptotically optimal. We also describe a variant of Mergesort with query complexity $O(wn\log\frac{n}{w})$ and total complexity $O(w^{2}n\log\frac{n}{w})$; an algorithm with the same query complexity was given by Faigle and Turn, but no efficient implementation of that algorithm is known. Both our sorting algorithms can be applied with negligible overhead to the more general problem of reconstructing transitive relations. We also consider two related problems: finding the minimal elements, and its generalization to finding the bottom k levels, called the k-selection problem. We give efficient deterministic and randomized algorithms for finding the minimal elements with query complexity and total complexity $O(wn)$. We provide matching lower bounds for the query complexity up to a factor of 2 and generalize the results to the k-selection problem. Finally, we present efficient algorithms for computing a linear extension of a poset and computing the heights of all elements.

computer science, theory & methods,mathematics, applied
Scalable Distributed String Sorting

Florian Kurpicz,Pascal Mehnert,Peter Sanders,Matthias Schimek

2024-04-25

Abstract:String sorting is an important part of tasks such as building index data structures. Unfortunately, current string sorting algorithms do not scale to massively parallel distributed-memory machines since they either have latency (at least) proportional to the number of processors $p$ or communicate the data a large number of times (at least logarithmic). We present practical and efficient algorithms for distributed-memory string sorting that scale to large $p$. Similar to state-of-the-art sorters for atomic objects, the algorithms have latency of about $p^{1/k}$ when allowing the data to be communicated $k$ times. Experiments indicate good scaling behavior on a wide range of inputs on up to 49152 cores. Overall, we achieve speedups of up to 5 over the current state-of-the-art distributed string sorting algorithms.

Data Structures and Algorithms
Dualheap Selection Algorithm: Efficient, Inherently Parallel and Somewhat Mysterious

Greg Sepesi

DOI: https://doi.org/10.48550/arXiv.0706.2155

2007-06-15

Abstract:An inherently parallel algorithm is proposed that efficiently performs selection: finding the K-th largest member of a set of N members. Selection is a common component of many more complex algorithms and therefore is a widely studied problem. Not much is new in the proposed dualheap selection algorithm: the heap data structure is from <a class="link-external link-http" href="http://J.W.J.Williams" rel="external noopener nofollow">this http URL</a>, the bottom-up heap construction is from R.W. Floyd, and the concept of a two heap data structure is from J.W.J. Williams and D.E. Knuth. The algorithm's novelty is limited to a few relatively minor implementation twists: 1) the two heaps are oriented with their roots at the partition values rather than at the minimum and maximum values, 2)the coding of one of the heaps (the heap of smaller values) employs negative indexing, and 3) the exchange phase of the algorithm is similar to a bottom-up heap construction, but navigates the heap with a post-order tree traversal. When run on a single processor, the dualheap selection algorithm's performance is competitive with quickselect with median estimation, a common variant of C.A.R. Hoare's quicksort algorithm. When run on parallel processors, the dualheap selection algorithm is superior due to its subtasks that are easily partitioned and innately balanced.

Data Structures and Algorithms,Computational Complexity,Distributed, Parallel, and Cluster Computing
TMS: An Novel Algorithm for Top-k Queries with Multi-Dimensional Selections on Massive Data

Xixian Han,Xianmin Liu,Jianzhong Li,Hong Gao

DOI: https://doi.org/10.7544/issn1000-1239.2017.20150615

2017-01-01

Abstract:In many applications,top-k query is an important operation to return a set of interesting points among potentially huge data space.Usually,a multi-dimensional selection is specified in top-k query.It is found that the existing algorithms cannot process the query on massive data efficiently.A sorted-list based algorithm TMS (top-k with multi-dimensional selection) is proposed to process top-k query with selection condition on massive data efficiently.TMS employs selection attribute lattice (SAL) of hierarchical structure to distribute tuples and obtains a horizontal partitioning of the table.Each partition is stored in column-oriented model and the lists of ranking attributes are arranged in descending order of attribute values.Given multi-dimensional selection,the relevant lattice cells are determined by SAL and this reduces the number of the retrieved tuples significantly.Double-sorting operation is devised to perform progressive selection evaluation.The efficient pruning is developed to discard the candidates which do not satisfy selection condition or score requirement.The extensive experimental results show that TMS has a significant performance advantage over the existing algorithms.
On Instance-Optimal Algorithms for a Generalization of Nuts and Bolts and Generalized Sorting

Mayank Goswami,Riko Jacob

2024-07-15

Abstract:We generalize the classical nuts and bolts problem to a setting where the input is a collection of $n$ nuts and $m$ bolts, and there is no promise of any matching pairs. It is not allowed to compare a nut directly with a nut or a bolt directly with a bolt, and the goal is to perform the fewest nut-bolt comparisons to discover the partial order between the nuts and bolts. We term this problem \emph{bipartite sorting}. We show that instances of bipartite sorting of the same size exhibit a wide range of complexity, and propose to perform a fine-grained analysis for this problem. We rule out straightforward notions of instance-optimality as being too stringent, and adopt a \emph{neighborhood-based} definition. Our definition may be of independent interest as a unifying lens for instance-optimal algorithms for other static problems existing in literature. This includes problems like sorting (Estivill-Castro and Woods, ACM Comput. Surv. 1992), convex hull (Afshani, Barbay and Chan, JACM 2017), adaptive joins (Demaine, López-Ortiz and Munro, SODA 2000), and the recent concept of universal optimality for graphs (Haeupler, Hladík, Rozhoň, Tarjan and Tětek, 2023). As our main result on bipartite sorting, we give a randomized algorithm that is within a factor of $O(\log ^3 (n+m))$ of being instance-optimal w.h.p., with respect to the neighborhood-based definition. As our second contribution, we generalize bipartite sorting to DAG sorting, when the underlying DAG is not necessarily bipartite. As an unexpected consequence of a simple algorithm for DAG sorting, we rule out a potential lower bound on the widely-studied problem of \emph{sorting with priced information}, posed by (Charikar, Fagin, Guruswami, Kleinberg, Raghavan and Sahai, STOC 2000).

Data Structures and Algorithms
Schedule Length and Reliability-Oriented Multi-Objective Scheduling for Distributed Computing

Guoquan Liu,Yifeng Zeng,Dong Li,Yingke Chen

DOI: https://doi.org/10.1007/s00500-014-1360-3

IF: 3.732

2014-01-01

Soft Computing

Abstract:Maximizing system reliability and minimizing schedule length are the two major objectives in scheduling a distributed computing system. These two objectives have been considered separately by most researchers, although more realistically they should be considered simultaneously. This paper addresses the problem by taking a multi-objective approach in scheduling. A Tabu search algorithm is proposed and two lateral interference schemes are used to distribute the Pareto optimal solutions along the Pareto front uniformly. Randomly generated directed acyclic graphs and a real application task graph are used to study the performance of the proposed algorithms. Experimental results show that for this problem lateral interference has no influence on the non-dominated solution number, but does benefit the uniform distribution of non-dominated solutions, irrespective of the computation method used to determine distances between the solutions.
An Algorithm For The Construction Of Concentrators From 2x2 Sorters

Shuo-Yen Robert Li,Gar Man Koo,Hui Li

DOI: https://doi.org/10.1090/dimacs/042/13

1998-01-01

Abstract:This paper presents polynomial-time algorithms for the construction of deterministic and internally non-blocking concentrators from multi-stage parallel cascades of 2 x 2 sorters. The approach is a generalization of "Fast Knockout" and "Sortout" techniques. The goal is to construct best known m-to-n networks of 2 x 2 sorters for m-to-n concentration, where m and n are within the practical range. The main criterion on the complexity of concentration is the number of stages in the cascade.
High-Performance and Flexible Parallel Algorithms for Semisort and Related Problems

Xiaojun Dong,Yunshu Wu,Zhongqi Wang,Laxman Dhulipala,Yan Gu,Yihan Sun

DOI: https://doi.org/10.1145/3558481.3591071

2023-04-20

Abstract:Semisort is a fundamental algorithmic primitive widely used in the design and analysis of efficient parallel algorithms. It takes input as an array of records and a function extracting a \emph{key} per record, and reorders them so that records with equal keys are contiguous. Since many applications only require collecting equal values, but not fully sorting the input, semisort is broadly applicable, e.g., in string algorithms, graph analytics, and geometry processing, among many other domains. However, despite dozens of recent papers that use semisort in their theoretical analysis and the existence of an asymptotically optimal parallel semisort algorithm, most implementations of these parallel algorithms choose to implement semisort by using comparison or integer sorting in practice, due to potential performance issues in existing semisort implementations. In this paper, we revisit the semisort problem, with the goal of achieving a high-performance parallel semisort implementation with a flexible interface. Our approach can easily extend to two related problems, \emph{histogram} and \emph{collect-reduce}. Our algorithms achieve strong speedups in practice, and importantly, outperform state-of-the-art parallel sorting and semisorting methods for almost all settings we tested, with varying input sizes, distribution, and key types. We also test two important applications with real-world data, and show that our algorithms improve the performance over existing approaches. We believe that many other parallel algorithm implementations can be accelerated using our results.

Data Structures and Algorithms

Multi-selection and Distributed Sorting

Distributed Privacy-Aware Fast Selection Algorithm for Large-Scale Data.

Synergistic Sorting, MultiSelection and Deferred Data Structures on MultiSets

Parallel External Selection Algorithm on Distributed Memory Systems

A Partitioning Selection Algorithm on Multiprocessors

High Performance Parallel Sort for Shared and Distributed Memory MIMD

Parallel Sorting by Approximate Splitting for Multi-core Processors

Distributed Semi-supervised Multi-label Classification with Quantized Communication

Data-centric workloads with MPI_Sort

Anytime Sorting Algorithms (Extended Version)

Parallel Multi-Deque Partition Dual-Deque Merge sorting algorithm using OpenMP

Array Sort: an Adaptive Sorting Algorithm on Multi-Thread

Multiobjective Feature Selection for Microarray Data Via Distributed Parallel Algorithms.

Sorting and Selection in Posets

Scalable Distributed String Sorting

Dualheap Selection Algorithm: Efficient, Inherently Parallel and Somewhat Mysterious

TMS: An Novel Algorithm for Top-k Queries with Multi-Dimensional Selections on Massive Data

On Instance-Optimal Algorithms for a Generalization of Nuts and Bolts and Generalized Sorting

Schedule Length and Reliability-Oriented Multi-Objective Scheduling for Distributed Computing

An Algorithm For The Construction Of Concentrators From 2x2 Sorters

High-Performance and Flexible Parallel Algorithms for Semisort and Related Problems