Constructions and bounds for separating hash families

Xiaolei Niu,Haitao Cao

DOI: https://doi.org/10.1016/j.disc.2018.06.018

IF: 0.961

2018-09-01

Discrete Mathematics

Abstract:In this paper, we present a new construction for strong separating hash families by using hypergraphs and obtain some optimal separating hash families. We also improve some previously known bounds of separating hash families.

mathematics

What problem does this paper attempt to address?

A tight bound on the size of certain separating hash families

Chuan Guo,Douglas R. Stinson

DOI: https://doi.org/10.48550/arXiv.1510.00293

2015-10-01

Information Theory

Abstract:In this paper, we present a new lower bound on the size of separating hash families of type {w_1^{q-1},w_2} where w_1 < w_2. Our result extends the paper by Guo et al. on binary frameproof codes. This bound compares well against known general bounds, and is especially useful when trying to bound the size of strong separating hash families. We also show that our new bound is tight by constructing hash families that meet the new bound with equality.
A Balanced Semi-Supervised Hashing Method for CBIR

Jianhui Zhou,Haiyan Fu,Xiangwei Kong

DOI: https://doi.org/10.1109/icip.2011.6116164

2011-01-01

Abstract:Hashing methods have attracted much attention in large scale image research in recent years, because they are not only fast, but also needing a little memory. This paper proposed a balanced semi-supervised hashing method by dividing image into several blocks. With the help of improved semi-supervised hashing, we obtain a short hash code of each block, which jointed together forms a hash code of an integrated image. In the improved semi-supervised hashing, the supervised information is completed by combining the similarity of image pairs and label information. Extensive experiments demonstrate that our method can get more balanced result between retrieval speed, saving storage of original data and retrieval accuracy in CBIR than the state-of-the-art hashing methods.
A New Linkless Hierarchical Structure Based on Perfect Hashing

Weiwei Duan,Jianxin Luo,Guiqiang Ni,Qi Hu,Yi Gao

DOI: https://doi.org/10.12783/dtcse/csae2017/17475

2017-01-01

Abstract:We explore using perfect spatial hashing to store octrees efficiently while needing no explicit parent-to-child pointers. Specifically, we design a new linkless hierarchical structure – one that encodes the absolute locations of the subdivided nodes using specific tokens and packs these nodes into hash tables using perfect hashing. Whereas prior methods have overhead in space and time due to the unused links and chain of the indirection pointers, our linkless hierarchical structure has lower storage requirements and has an efficient random accessibility on the GPUs. The performance of our scheme is demonstrated on different 2D/3D datasets together with some classical algorithms.
Distributed Discrete Hashing by Equivalent Continuous Formulation.

Shengnan Wang,Chunguang Li,Hui-Liang Shen

DOI: https://doi.org/10.1109/tsipn.2020.2975356

2020-01-01

IEEE Transactions on Signal and Information Processing over Networks

Abstract:Hashing based approximate nearest neighbor search has attracted considerable attention in various fields. Most of the existing hashing methods are centralized, which cannot be used for many large-scale applications with the data stored or collected in a distributed manner. In this article, we consider the distributed hashing problem. The main difficulty of hashing is brought by its inherent binary constraints, which makes the problem generally NP-hard. Most of the existing distributed hashing methods chose to relax the problem by dropping the binary constraints. However, such a manner will bring additional quantization error, which makes the binary codes less effective. In this paper, we propose a novel distributed discrete hashing method, which learns effective hash codes without using any relaxations. Specifically, we give a method to transform the discrete hashing problem into an equivalent distributed continuous optimization problem. After transformation, we devise a distributed discrete hashing (dDH) algorithm based on the idea of DC programming to solve the problem. To obtain more efficient hash codes, we further add bits balance and uncorrelation constraints to the hashing problem, and we also propose a distributed constrained discrete hashing algorithm (dCDH) to solve this problem. Extensive experiments are provided to show the superiority of the proposed methods.
Constructions of Strongly Regular Cayley Graphs and Skew Hadamard Difference Sets from Cyclotomic Classes

Tao Feng,Koji Momihara,Qing Xiang

DOI: https://doi.org/10.1007/s00493-014-2895-8

2014-01-01

COMBINATORICA

Abstract:We find new constructions of infinite families of skew Hadamard difference sets in elementary abelian groups under the assumption of the existence of cyclotomic strongly regular graphs. Our construction is based on choosing cyclotomic classes in finite fields.
Ordered Equivalence Class Partition On Hash

Jiang Yunliang,Fan Jing,Liu Yong

2010-01-01

Abstract:Generally speaking, performance analysis of the algorithm is only relevant to the size of data set. But in practice, if not considering the data distribution, it affect the adaptability analysis on algorithm. Rough set theory, proposed by Pawlak in the early 1980s, is a mathematical tool in computer application. It has been successfully applied in machine learning, pattern recognition, expert systems, data analysis, and so on. In rough set theory, data are presented as a table. After ordering the two-dimensional table, it is easy to obtain the equivalence class. The most important reason is that the attributes and the value range in table determine the equivalence class, and two concepts saturation and concentration can describe the distribution of attribute value. In this paper, we propose a new concept of ordered equivalence class which is an extension of equivalence class. In fact, ordering the two-dimensional table is the main step to get ordered equivalence class. Usually the time complexity of it is O(vertical bar U vertical bar vertical bar R vertical bar log vertical bar U vertical bar,) where vertical bar U vertical bar is the number of entries and vertical bar R vertical bar is the number of attributes. Paper [1] make a profound analysis reduce the time complexity to O(vertical bar U vertical bar x(vertical bar R vertical bar + log vertical bar U vertical bar)) In this paper, we introduce a new algorithm based on hash with the time complexity O(vertical bar U vertical bar vertical bar R vertical bar) to obtain the ordered equivalence class. Drawing a comparison, especially on the condition that the saturation and concentration are both high, the hash algorithm is more efficient than in paper[1]
Explicit Constructions of Perfect Hash Families from Algebraic Curves over Finite Fields

Huaxiong Wang,Chaoping Xing

DOI: https://doi.org/10.1006/jcta.2000.3068

2001-01-01

Abstract:Let A be a set of order n and B be a set of order m. An (n, m, w)-perfect hash family is a set H of functions from A to B such that for any X⊆A with |X|=w, there exists an element h∈H such that h is one-to-one when restricted to X. Perfect hash families have many applications to computer science, such as database management, circuit complexity theory and cryptography. In this paper, we provide explicit constructions of perfect hash families based on algebraic curves over finite fields. In particular, using the Garcia–Stichtenoth curves, we obtain infinite classes of (n, m, w)-perfect hash families with |H|=O(logn) for fixed m and w, which are among the most efficient explicit constructions for perfect hash families known in the literature. We also exhibit examples to show the efficiency of the new constructions and their applications to the constructions of cover-free families.
Discrete Double-Bit Hashing

Shengnan Wang,Chunguang Li

DOI: https://doi.org/10.1109/tbdata.2019.2946616

2019-01-01

IEEE Transactions on Big Data

Abstract:Hashing has been widely used for nearest neighbors search over big data. Hashing encodes high dimensional data points into binary codes. Most hashing methods use the single-bit quantization (SBQ) strategy for coding the data. However, this strategy often encodes neighboring points into totally different bits. Recently, a double-bit quantization (DBQ) strategy was proposed, which can better preserve the similarity of the data. The hashing problems are generally NP-hard, due to the discrete constraints. For tractability, some relaxation methods were proposed by discarding the discrete constraints. However, such a manner makes the hash codes less effective, due to the large quantization error. To obtain high-quality hash codes, some discrete hashing methods were proposed, which directly solve the hashing problem without any relaxations. However, the existing discrete hashing methods can only deal with single-bit hashing. In this paper, we propose a discrete hashing method to solve double-bit hashing problems. To address the difficulty brought by the discrete constraints, we propose a method to transform the discrete hashing problem into an equivalent continuous optimization problem. Then, we devise algorithms based on DC (difference of convex functions) programming to solve the problem. Numerical experiments are provided to show the superiority of the proposed methods.
Deterministic Coupon Collection and Better Strong Dispersers.

Raghu Meka,Omer Reingold,Yuan Zhou

DOI: https://doi.org/10.4230/lipics.approx-random.2014.872

2014-01-01

Abstract:Hashing is one of the main techniques in data processing and algorithm design for very large data sets. While random hash functions satisfy most desirable properties, it is often too expensive to store a fully random hash function. Motivated by this, much attention has been given to designing small families of hash functions suitable for various applications. In this work, we study the question of designing space-efficient hash families H = {h:[U] -> [N]} with the natural property of 'covering': H is said to be covering if any set of Omega(N log N) distinct items from the universe (the coupon-collector limit) are hashed to cover all N bins by most hash functions in H. We give an explicit covering family H of size poly(N) (which is optimal), so that hash functions in H can be specified efficiently by O(log N) bits. We build covering hash functions by drawing a connection to dispersers, which are quite well-studied and have a variety of applications themselves. We in fact need strong dispersers and we give new constructions of strong dispersers which may be of independent interest. Specifically, we construct strong dispersers with optimal entropy loss in the high min-entropy, but very small error (poly(n)/2^n for n bit sources) regimes. We also provide a strong disperser construction with constant error but for any min-entropy. Our constructions achieve these by using part of the source to replace seed from previous non-strong constructions in surprising ways. In doing so, we take two of the few constructions of dispersers with parameters better than known extractors and make them strong.
Equivalent Continuous Formulation of General Hashing Problem

Shengnan Wang,Chunguang Li,Hui-Liang Shen

DOI: https://doi.org/10.1109/tcyb.2019.2894020

IF: 11.8

2019-01-01

IEEE Transactions on Cybernetics

Abstract:Hashing-based approximate nearest neighbors search has attracted broad research interest, due to its low computational cost and fast retrieval speed. The hashing technique maps the data points into binary codes and, meanwhile, preserves the similarity in the original space. Generally, we need to solve a discrete optimization problem to learn the binary codes and hash functions, which is NP-hard. In the literature, most hashing methods choose to solve a relaxed problem by discarding the discrete constraints. However, such a relaxation scheme will cause large quantization error, which makes the learned binary codes less effective. In this paper, we present an equivalent continuous formulation of the discrete hashing problem. Specifically, we show that the discrete hashing problem can be transformed into a continuous optimization problem without any relaxations, while the transformed continuous optimization problem has the same optimal solutions and the same optimal value as the original discrete hashing problem. After transformation, the continuous optimization methods can be applied. We devise the algorithms based on the idea of DC (difference of convex functions) programming to solve this problem. The proposed continuous hashing scheme can be easily applied to the existing hashing models, including both supervised and unsupervised hashing. We evaluate the proposed method on several benchmarks and the results show the superiority of the proposed method compared with the state-of-the-art hashing methods.
Construction and security analysis of hash algorithm based on stream cipher

Huafei Zhu,Lixin Ran,Yumin Wang

IF: 1.019

1999-01-01

Chinese Journal of Electronics

Abstract:Cryptographic hash functions are very important for cryptographic protocols such as signature scheme and message authentication. Based on the pioneer work of X. Lai et al., a new framework of hash algorithm is developed in this paper. In our hash scheme, an extra pseudorandom keystream generator is not needed which may be more conveniently for practical use. We have proved that the security of the new hash scheme is equivalent to that of feedback shift registers (FSRs) controlled by 2m - bit security keys.
Parallel and External-Memory Construction of Minimal Perfect Hash Functions with PTHash

Giulio Ermanno Pibiri,Roberto Trani

2023-08-06

Abstract:A function $f : U \to \{0,\ldots,n-1\}$ is a minimal perfect hash function for a set $S \subseteq U$ of size $n$, if $f$ bijectively maps $S$ into the first $n$ natural numbers. These functions are important for many practical applications in computing, such as search engines, computer networks, and databases. Several algorithms have been proposed to build minimal perfect hash functions that: scale well to large sets, retain fast evaluation time, and take very little space, e.g., 2 - 3 bits/key. PTHash is one such algorithm, achieving very fast evaluation in compressed space, typically several times faster than other techniques. In this work, we propose a new construction algorithm for PTHash enabling: (1) multi-threading, to either build functions more quickly or more space-efficiently, and (2) external-memory processing to scale to inputs much larger than the available internal memory. Only few other algorithms in the literature share these features, despite of their big practical impact. We conduct an extensive experimental assessment on large real-world string collections and show that, with respect to other techniques, PTHash is competitive in construction time and space consumption, but retains 2 - 6$\times$ better lookup time.

Data Structures and Algorithms
Construction of Strong-Uniform Fuzzy Partitions of Arbitrary Dimensions

Zhi Zeng,Ting Wang

2022-01-01

Abstract: Strong-uniform fuzzy partition is necessary for the accuracy of fuzzy partition-based histograms. Most previous research focused on constructing one-dimensional strong-uniform fuzzy partitions. While to the best of our knowledge, few have been reported for high-dimensional cases. In order to fill this theoretical vacancy, this paper proves the existence of high-dimensional strong-uniform fuzzy partitions via proposing an analytic formula to construct strong-uniform fuzzy partitions of arbitrary dimensions.
Linear Hashing with $\ell_\infty$ guarantees and two-sided Kakeya bounds

Manik Dhar,Zeev Dvir

DOI: https://doi.org/10.46298/theoretics.24.8

2024-03-30

Abstract:We show that a randomly chosen linear map over a finite field gives a good hash function in the $\ell_\infty$ sense. More concretely, consider a set $S \subset \mathbb{F}_q^n$ and a randomly chosen linear map $L : \mathbb{F}_q^n \to \mathbb{F}_q^t$ with $q^t$ taken to be sufficiently smaller than $ |S|$. Let $U_S$ denote a random variable distributed uniformly on $S$. Our main theorem shows that, with high probability over the choice of $L$, the random variable $L(U_S)$ is close to uniform in the $\ell_\infty$ norm. In other words, {\em every} element in the range $\mathbb{F}_q^t$ has about the same number of elements in $S$ mapped to it. This complements the widely-used Leftover Hash Lemma (LHL) which proves the analog statement under the statistical, or $\ell_1$, distance (for a richer class of functions) as well as prior work on the expected largest 'bucket size' in linear hash functions [ADMPT99]. By known bounds from the load balancing literature [RS98], our results are tight and show that linear functions hash as well as trully random function up to a constant factor in the entropy loss. Our proof leverages a connection between linear hashing and the finite field Kakeya problem and extends some of the tools developed in this area, in particular the polynomial method.

Combinatorics,Computational Complexity,Cryptography and Security
Circular External Difference Families: Construction and Non-Existence

Huawei Wu,Jing Yang,Keqin Feng

2023-10-30

Abstract:The circular external difference family and its strong version, which themselves are of independent combinatorial interest, were proposed as variants of the difference family to construct new unconditionally secure non-malleable threshold schemes. In this paper, we present new results regarding the construction and non-existence of (strong) circular external difference families, thereby solving several open problems on this topic.

Combinatorics,Information Theory
Balanced Hashing

Dahan Gong,Chaoqun Chu,Kai Chen,Yuchen Guo,Guiguang Ding

DOI: https://doi.org/10.1049/cje.2019.08.003

IF: 1.019

2019-01-01

Chinese Journal of Electronics

Abstract:Hashing for nearest neighbor search has attracted considerable interest recently given its efficiency in speed and storage. Many methods follow a projection-quantization framework which firstly projects original data into low-dimensional compact space and secondly quantifies each projected dimension to 1 bit by thresholding. The variance of projected dimensions,however, may vary a lot so that quantifying them equivalently degrades the searching performance. In this paper, we put forward a novel method, dubbed Balanced hashing（BH）, which finds adjustment functions to reproject the data such that the variance of dimensions can be balanced by directly and explicitly maximizing the degree of balance of data, while preserving important properties. Experiments on benchmarks demonstrate that BH can outperform several state-of-the-art methods.
An improvement of degree-based hashing (DBH) graph partition method, using a novel metric

Anna Mastikhina,Oleg Senkevich,Dmitry Sirotkin,Danila Demin,Stanislav Moiseev

2024-04-11

Abstract:This paper examines the graph partition problem and introduces a new metric, MSIDS (maximal sum of inner degrees squared). We establish its connection to the replication factor (RF) optimization, which has been the main focus of theoretical work in this field. Additionally, we propose a new partition algorithm, DBH-X, based on the DBH partitioner. We demonstrate that DBH-X significantly improves both the RF and MSIDS, compared to the baseline DBH algorithm. In addition, we provide test results that show the runtime acceleration of GraphX-based PageRank and Label propagation algorithms.

Data Structures and Algorithms
Stability through non-shadows

Jun Gao,Hong Liu,Zixiang Xu

2023-06-25

Abstract:We study families $\mathcal{F}\subseteq 2^{[n]}$ with restricted intersections and prove a conjecture of Snevily in a stronger form for large $n$. We also obtain stability results for Kleitman's isodiametric inequality and families with bounded set-wise differences. Our proofs introduce a new twist to the classical linear algebra method, harnessing the non-shadows of $\mathcal{F}$, which may be of independent interest.

Combinatorics
Practical Hashing Function for URLs Set

XIAO Ming-zhong,MIN Bo-nan,WANG Jia-chong,DAI Ya-fei

DOI: https://doi.org/10.3969/j.issn.1000-1220.2006.03.033

2006-01-01

Abstract:URL hashing is found many applications in Web research. We propose a hashing function for large scale URLs set and find it has better uniformity and stability than the other two (HfIp and hf) through three experiments of the large scale. It is a variation of the well-known function (ELFhash) and is recommended used in the applications of needing to hash URLs. Moreover, it has low time cost and almost performance compared with MD5 and SHA-1 so that we think it is more practical than the other. Finally, some future works are given.
Fast and Accurate Hashing Via Iterative Nearest Neighbors Expansion.

Zhongming Jin,Debing Zhang,Yao Hu,Shiding Lin,Deng Cai,Xiaofei He

DOI: https://doi.org/10.1109/tcyb.2014.2302018

IF: 11.8

2014-01-01

IEEE Transactions on Cybernetics

Abstract:Recently, the hashing techniques have been widely applied to approximate the nearest neighbor search problem in many real applications. The basic idea of these approaches is to generate binary codes for data points which can preserve the similarity between any two of them. Given a query, instead of performing a linear scan of the entire data base, the hashing method can perform a linear scan of the points whose hamming distance to the query is not greater than r(h), where r(h) is a constant. However, in order to find the true nearest neighbors, both the locating time and the linear scan time are proportional to O(Sigma(rh)(i=0) ((c)(i)))) (c is the code length), which increase exponentially as r(h) increases. To address this limitation, we propose a novel algorithm named iterative expanding hashing in this paper, which builds an auxiliary index based on an offline constructed nearest neighbor table to avoid large r(h). This auxiliary index can be easily combined with all the traditional hashing methods. Extensive experimental results over various real large-scale datasets demonstrate the superiority of the proposed approach.

Constructions and bounds for separating hash families

A tight bound on the size of certain separating hash families

A Balanced Semi-Supervised Hashing Method for CBIR

A New Linkless Hierarchical Structure Based on Perfect Hashing

Distributed Discrete Hashing by Equivalent Continuous Formulation.

Constructions of Strongly Regular Cayley Graphs and Skew Hadamard Difference Sets from Cyclotomic Classes

Ordered Equivalence Class Partition On Hash

Explicit Constructions of Perfect Hash Families from Algebraic Curves over Finite Fields

Discrete Double-Bit Hashing

Deterministic Coupon Collection and Better Strong Dispersers.

Equivalent Continuous Formulation of General Hashing Problem

Construction and security analysis of hash algorithm based on stream cipher

Parallel and External-Memory Construction of Minimal Perfect Hash Functions with PTHash

Construction of Strong-Uniform Fuzzy Partitions of Arbitrary Dimensions

Linear Hashing with $\ell_\infty$ guarantees and two-sided Kakeya bounds

Circular External Difference Families: Construction and Non-Existence

Balanced Hashing

An improvement of degree-based hashing (DBH) graph partition method, using a novel metric

Stability through non-shadows

Practical Hashing Function for URLs Set

Fast and Accurate Hashing Via Iterative Nearest Neighbors Expansion.