Optimizing Learned Bloom Filters by Sandwiching

Michael Mitzenmacher

DOI: https://doi.org/10.48550/arXiv.1803.01474

2018-03-05

Data Structures and Algorithms

Abstract:We provide a simple method for improving the performance of the recently introduced learned Bloom filters, by showing that they perform better when the learned function is sandwiched between two Bloom filters.

What problem does this paper attempt to address?

A Model for Learned Bloom Filters, and Optimizing by Sandwiching

Michael Mitzenmacher

DOI: https://doi.org/10.48550/arXiv.1901.00902

2019-01-04

Abstract:Recent work has suggested enhancing Bloom filters by using a pre-filter, based on applying machine learning to determine a function that models the data set the Bloom filter is meant to represent. Here we model such learned Bloom filters,, with the following outcomes: (1) we clarify what guarantees can and cannot be associated with such a structure; (2) we show how to estimate what size the learning function must obtain in order to obtain improved performance; (3) we provide a simple method, sandwiching, for optimizing learned Bloom filters; and (4) we propose a design and analysis approach for a learned Bloomier filter, based on our modeling approach.

Machine Learning,Databases
A Model for Learned Bloom Filters and Related Structures

Michael Mitzenmacher

DOI: https://doi.org/10.48550/arXiv.1802.00884

2018-02-03

Abstract:Recent work has suggested enhancing Bloom filters by using a pre-filter, based on applying machine learning to model the data set the Bloom filter is meant to represent. Here we model such learned Bloom filters, clarifying what guarantees can and cannot be associated with such a structure.

Data Structures and Algorithms
A Critical Analysis of Classifier Selection in Learned Bloom Filters

Dario Malchiodi,Davide Raimondi,Giacomo Fumagalli,Raffaele Giancarlo,Marco Frasca

DOI: https://doi.org/10.48550/arXiv.2211.15565

2022-11-29

Abstract:Learned Bloom Filters, i.e., models induced from data via machine learning techniques and solving the approximate set membership problem, have recently been introduced with the aim of enhancing the performance of standard Bloom Filters, with special focus on space occupancy. Unlike in the classical case, the "complexity" of the data used to build the filter might heavily impact on its performance. Therefore, here we propose the first in-depth analysis, to the best of our knowledge, for the performance assessment of a given Learned Bloom Filter, in conjunction with a given classifier, on a dataset of a given classification complexity. Indeed, we propose a novel methodology, supported by software, for designing, analyzing and implementing Learned Bloom Filters in function of specific constraints on their multi-criteria nature (that is, constraints involving space efficiency, false positive rate, and reject time). Our experiments show that the proposed methodology and the supporting software are valid and useful: we find out that only two classifiers have desirable properties in relation to problems with different data complexity, and, interestingly, none of them has been considered so far in the literature. We also experimentally show that the Sandwiched variant of Learned Bloom filters is the most robust to data complexity and classifier performance variability, as well as those usually having smaller reject times. The software can be readily used to test new Learned Bloom Filter proposals, which can be compared with the best ones identified here.

Machine Learning,Artificial Intelligence
Optimizing Collections of Bloom Filters within a Space Budget

Gabriel Mersy,Zhuo Wang,Stavros Sintos,Sanjay Krishnan

DOI: https://doi.org/10.14778/3681954.3682020

IF: 2.5

2024-07-01

Proceedings of the VLDB Endowment

Abstract:With a single Bloom filter, one can approximately answer set membership queries within a space budget. Practical systems often use collections of Bloom filters to facilitate applications such as data skipping, sideways information passing, and network filtering. While the optimal space-to-accuracy allocation is well-understood for a single filter, jointly optimizing how space is used across a collection of filters is yet to be studied. We pose this problem in the following way: (1) let's assume that each Bloom filter has some likelihood of being queried, and (2) given knowledge of this likelihood, how do we allocate space to minimize the expected false positive rate? In other words, "hot" filters are allocated more space, and "cold" filters are allocated less space. In this paper, we show how to solve this optimization problem. We first develop the concept of a "truncated" Bloom filter and theoretically analyze its false positive rate. We then formulate an optimization problem for a collection of truncated Bloom filters that minimizes the false positive rate across a utility distribution while meeting a strict space budget. Next, we show that the problem is convex and find a fast relaxation. Lastly, we apply our method to data skipping and full-text search, demonstrating its effectiveness across the range of possible space budgets when compared to the state of the art.

computer science, information systems, theory & methods
Fast Construction of Partitioned Learned Bloom Filter with Theoretical Guarantees

Atsuki Sato,Yusuke Matsui

2024-10-17

Abstract:Bloom filter is a widely used classic data structure for approximate membership queries. Learned Bloom filters improve memory efficiency by leveraging machine learning, with the partitioned learned Bloom filter (PLBF) being among the most memory-efficient variants. However, PLBF suffers from high computational complexity during construction, specifically $O(N^3k)$, where $N$ and $k$ are hyperparameters. In this paper, we propose three methods: fast PLBF, fast PLBF++, and fast PLBF#, that reduce the construction complexity to $O(N^2k)$, $O(Nk \log N)$, and $O(Nk \log k)$, respectively. Fast PLBF preserves the original PLBF structure and memory efficiency. Although fast PLBF++ and fast PLBF# may have different structures, we theoretically prove they are equivalent to PLBF under ideal data distribution. Furthermore, we theoretically bound the difference in memory efficiency between PLBF and fast PLBF++ for non-ideal scenarios. Experiments on real-world datasets demonstrate that fast PLBF, fast PLBF++, and fast PLBF# are up to 233, 761, and 778 times faster to construct than original PLBF, respectively. Additionally, fast PLBF maintains the same data structure as PLBF, and fast PLBF++ and fast PLBF# achieve nearly identical memory efficiency.

Data Structures and Algorithms
Stable Learned Bloom Filters for Data Streams

Qiyu Liu,Libin Zheng,Yanyan Shen,Lei Chen

DOI: https://doi.org/10.14778/3407790.3407830

IF: 2.5

2020-01-01

Proceedings of the VLDB Endowment

Abstract:Bloom filter and its variants are elegant space-efficient probabilistic data structures for approximate set membership queries. It has been recently shown that the space cost of Bloom filters can be significantly reduced via a combination with pre-trained machine learning models, named Learned Bloom filters (LBF). LBF eases the space requirement of a Bloom filter by undertaking part of the queries using a classifier. However, current LBF structures generally target a static member set. Their performances would inevitably decay when there is a member update on the set, while this update requirement is not uncommon for real-world data streaming applications such as duplicate item detection, malicious URL checking, and web caching. To adapt LBF to data streams, we propose the Stable Learned Bloom Filters (SLBF) which addresses the performance decay issue on intensive insertion workloads by combining classifier with updatable backup filters. Specifically, we propose two SLBF structures, Single SLBF (s-SLBF) and Grouping SLBF (g-SLBF). The theoretical analysis on these two structures shows that the expected false positive rate (FPR) of SLBF is asymptotically a constant over the insertion of new members. Extensive experiments on real-world datasets show that SLBF introduces a similar level of false negative rate (FNR) but yields a better FPR/storage trade-off compared with the state-of-the-art (non-learned) Bloom filters optimized on data streams.
Fast Partitioned Learned Bloom Filter

Atsuki Sato,Yusuke Matsui

2023-10-28

Abstract:A Bloom filter is a memory-efficient data structure for approximate membership queries used in numerous fields of computer science. Recently, learned Bloom filters that achieve better memory efficiency using machine learning models have attracted attention. One such filter, the partitioned learned Bloom filter (PLBF), achieves excellent memory efficiency. However, PLBF requires a $O(N^3k)$ time complexity to construct the data structure, where $N$ and $k$ are the hyperparameters of PLBF. One can improve memory efficiency by increasing $N$, but the construction time becomes extremely long. Thus, we propose two methods that can reduce the construction time while maintaining the memory efficiency of PLBF. First, we propose fast PLBF, which can construct the same data structure as PLBF with a smaller time complexity $O(N^2k)$. Second, we propose fast PLBF++, which can construct the data structure with even smaller time complexity $O(Nk\log N + Nk^2)$. Fast PLBF++ does not necessarily construct the same data structure as PLBF. Still, it is almost as memory efficient as PLBF, and it is proved that fast PLBF++ has the same data structure as PLBF when the distribution satisfies a certain constraint. Our experimental results from real-world datasets show that (i) fast PLBF and fast PLBF++ can construct the data structure up to 233 and 761 times faster than PLBF, (ii) fast PLBF can achieve the same memory efficiency as PLBF, and (iii) fast PLBF++ can achieve almost the same memory efficiency as PLBF. The codes are available at <a class="link-external link-https" href="https://github.com/atsukisato/FastPLBF" rel="external noopener nofollow">this https URL</a> .

Data Structures and Algorithms
Optimizing Bloom Filter: Challenges, Solutions, and Comparisons

Lailong Luo,Deke Guo,Richard T.B. Ma,Ori Rottenstreich,Xueshan Luo

DOI: https://doi.org/10.48550/arXiv.1804.04777

2019-01-07

Abstract:Bloom filter (BF) has been widely used to support membership query, i.e., to judge whether a given element x is a member of a given set S or not. Recent years have seen a flourish design explosion of BF due to its characteristic of space-efficiency and the functionality of constant-time membership query. The existing reviews or surveys mainly focus on the applications of BF, but fall short in covering the current trends, thereby lacking intrinsic understanding of their design philosophy. To this end, this survey provides an overview of BF and its variants, with an emphasis on the optimization techniques. Basically, we survey the existing variants from two dimensions, i.e., performance and generalization. To improve the performance, dozens of variants devote themselves to reducing the false positives and implementation costs. Besides, tens of variants generalize the BF framework in more scenarios by diversifying the input sets and enriching the output functionalities. To summarize the existing efforts, we conduct an in-depth study of the existing literature on BF optimization, covering more than 60 variants. We unearth the design philosophy of these variants and elaborate how the employed optimization techniques improve BF. Furthermore, comprehensive analysis and qualitative comparison are conducted from the perspectives of BF components. Lastly, we highlight the future trends of designing BFs. This is, to the best of our knowledge, the first survey that accomplishes such goals.

Data Structures and Algorithms
Adversary Resilient Learned Bloom Filters

Allison Bishop,Hayder Tirmazi

2024-10-14

Abstract:Creating an adversary resilient construction of the Learned Bloom Filter with provable guarantees is an open problem. We define a strong adversarial model for the Learned Bloom Filter. Our adversarial model extends an existing adversarial model designed for the Classical (i.e not ``Learned'') Bloom Filter by prior work and considers computationally bounded adversaries that run in probabilistic polynomial time (PPT). Using our model, we construct an adversary resilient variant of the Learned Bloom Filter called the Downtown Bodega Filter. We show that: if pseudo-random permutations exist, then an Adversary Resilient Learned Bloom Filter may be constructed with $2\lambda$ extra bits of memory and at most one extra pseudo-random permutation in the critical path. We construct a hybrid adversarial model for the case where a fraction of the query workload is chosen by an adversary. We show realistic scenarios where using the Downtown Bodega Filter gives better performance guarantees compared to alternative approaches in this hybrid model.

Cryptography and Security,Data Structures and Algorithms
A Pareto Optimal Bloom Filter Family with Hash Adaptivity

Meng Li,Rongbiao Xie,Deyi Chen,Haipeng Dai,Rong Gu,He Huang,Wanchun Dou,Guihai Chen

DOI: https://doi.org/10.1007/s00778-022-00755-z

2022-01-01

The VLDB Journal

Abstract:Bloom filter is a compact memory-efficient probabilistic data structure supporting membership testing, i.e., to check whether an element is in a given set. However, as Bloom filter maps each element with random hash functions, little flexibility is provided even if the information of negative keys (elements are not in the set) is available, especially when the misidentification of negative keys brings different costs. The problem worsens when the hash functions are non-uniform, i.e., mapping each element into Bloom filter non-uniformly. To address the above problem, we propose a new hash adaptive Bloom filter (HABF) that supports customizing hash functions for keys. Besides, we propose a filter family, including f-HABF (fast hashing version), c-HABF (cache-friendly version), and s-HABF (stacked version). We show that HABF family is Pareto optimal among all comparison filters in terms of accuracy and query latency. We conduct extensive experiments on representative datasets, and the results show that HABF family outperforms the standard Bloom filter and its cutting-edge variants on the whole in terms of accuracy, construction/query time, and memory space consumption. All the source codes are available in our source codes (https://github.com/njulands/HashAdaptiveBF).
Retouched Bloom Filters: Allowing Networked Applications to Flexibly Trade Off False Positives Against False Negatives

Benoit Donnet,Bruno Baynat,Timur Friedman

DOI: https://doi.org/10.48550/arXiv.cs/0607038

2006-12-01

Abstract:Where distributed agents must share voluminous set membership information, Bloom filters provide a compact, though lossy, way for them to do so. Numerous recent networking papers have examined the trade-offs between the bandwidth consumed by the transmission of Bloom filters, and the error rate, which takes the form of false positives, and which rises the more the filters are compressed. In this paper, we introduce the retouched Bloom filter (RBF), an extension that makes the Bloom filter more flexible by permitting the removal of selected false positives at the expense of generating random false negatives. We analytically show that RBFs created through a random process maintain an overall error rate, expressed as a combination of the false positive rate and the false negative rate, that is equal to the false positive rate of the corresponding Bloom filters. We further provide some simple heuristics and improved algorithms that decrease the false positive rate more than than the corresponding increase in the false negative rate, when creating RBFs. Finally, we demonstrate the advantages of an RBF over a Bloom filter in a distributed network topology measurement application, where information about large stop sets must be shared among route tracing monitors.

Networking and Internet Architecture
Bloofi: Multidimensional Bloom filters

Adina Crainiceanu,Daniel Lemire

DOI: https://doi.org/10.1016/j.is.2015.01.002

IF: 3.18

2015-12-01

Information Systems

Abstract:Bloom filters are probabilistic data structures commonly used for approximate membership problems in many areas of Computer Science (networking, distributed systems, databases, etc.). With the increase in data size and distribution of data, problems arise where a large number of Bloom filters are available, and all of them need to be searched for potential matches. As an example, in a federated cloud environment, each cloud provider could encode the information using Bloom filters and share the Bloom filters with a central coordinator. The problem of interest is not only whether a given element is in any of the sets represented by the Bloom filters, but also which of the existing sets contain the given element. This problem cannot be solved by just constructing a Bloom filter on the union of all the sets. Instead, we effectively have a multidimensional Bloom filter problem: given an element, we wish to receive a list of candidate sets where the element might be.To solve this problem, we consider three alternatives. Firstly, we can naively check many Bloom filters. Secondly, we propose to organize the Bloom filters in a hierarchical index structure akin to a B+ tree that we call Bloofi. Finally, we propose another data structure that packs the Bloom filters in such a way as to exploit bit-level parallelism, which we call Flat-Bloofi.Our theoretical and experimental results show that Bloofi and Flat-Bloofi provide scalable and efficient solutions alternatives to search through a large number of Bloom filters.

computer science, information systems
Invertible Bloom Lookup Tables

Michael T. Goodrich,Michael Mitzenmacher

DOI: https://doi.org/10.48550/arXiv.1101.2245

2015-10-04

Abstract:We present a version of the Bloom filter data structure that supports not only the insertion, deletion, and lookup of key-value pairs, but also allows a complete listing of its contents with high probability, as long the number of key-value pairs is below a designed threshold. Our structure allows the number of key-value pairs to greatly exceed this threshold during normal operation. Exceeding the threshold simply temporarily prevents content listing and reduces the probability of a successful lookup. If later entries are deleted to return the structure below the threshold, everything again functions appropriately. We also show that simple variations of our structure are robust to certain standard errors, such as the deletion of a key without a corresponding insertion or the insertion of two distinct values for a key. The properties of our structure make it suitable for several applications, including database and networking applications that we highlight.

Data Structures and Algorithms,Databases
Reducing the number of Bloom filters

qingge gong,tong yang,hongwei tong,kai shi,jinghui li,xianyan wu

DOI: https://doi.org/10.1109/PIC.2014.6972399

2014-01-01

Abstract:Bloom Filters have been applied in many fields, including data base, network management, computer network, and computer communication etc., owing to its fast membership query and memory efficiency. In network field, Bloom Filters were used to lookup the forwarding tables in backbone routers and Data Centre switches. These solutions all build many Bloom Filters to achieve fast lookup. The main shortcoming of these solution is: too many Bloom Filters require too many hash computations and memory accesses, and the variety of Bloom Filter sizes poses challenges to system design and probably degrades the system performance. To address this issues, we proposed Set absorption algorithm to reduce the number of Bloom Filters, while balancing the size of Bloom Filters. Experimental results showed that after using our algorithm, better performance of the Bloom Filter-based solutions was obtained.
On the Evolutionary of Bloom Filter False Positives - An Information Theoretical Approach to Optimizing Bloom Filter Parameters

Zhuochen Fan,Gang Wen,Zhipeng Huang,Yang Zhou,Qiaobin Fu,Tong Yang,Alex X. Liu,Bin Cui

DOI: https://doi.org/10.1109/tkde.2022.3200045

IF: 9.235

2022-01-01

IEEE Transactions on Knowledge and Data Engineering

Abstract:The fundamental issue of how to calculate the false positive probability of widely used Bloom Filters (BF), from which the conventional wisdom is to derive the optimal value of $k$k, remains elusive. Since Bloom gave the false positive formula in 1970, in 2008, Bose et al. pointed out that Bloom's formula is flawed; and in 2010, Christensen et al. pointed out that Bose's formula is also flawed and gave another formula. Although Christensen's formula is perfectly accurate, it is time-consuming and impossible to calculate the optimal value of $k$k. Based on the following observation: for a BF with $m$m bits and $n$n elements, if and only if its entropy is the largest, its false positive probability is the smallest, we propose the first approach to calculating the optimal $k$k without any false positive formula. Furthermore, we propose a new and more accurate upper bound for the false positive probability. When the size of a Bloom Filter becomes infinitely large, our upper bound turns equal to the lower bound, which becomes Bloom's formula and deepens our understanding towards it. Besides, we derive the bounds of correct rate of Counting Bloom Filters (CBFs) by applying our proposed formulas about BFs to them.

computer science, information systems, artificial intelligence,engineering, electrical & electronic
On the Choice of General Purpose Classifiers in Learned Bloom Filters: An Initial Analysis Within Basic Filters

Giacomo Fumagalli,Davide Raimondi,Raffaele Giancarlo,Dario Malchiodi,Marco Frasca

DOI: https://doi.org/10.48550/arXiv.2112.06563

2021-12-13

Abstract:Bloom Filters are a fundamental and pervasive data structure. Within the growing area of Learned Data Structures, several Learned versions of Bloom Filters have been considered, yielding advantages over classic Filters. Each of them uses a classifier, which is the Learned part of the data structure. Although it has a central role in those new filters, and its space footprint as well as classification time may affect the performance of the Learned Filter, no systematic study of which specific classifier to use in which circumstances is available. We report progress in this area here, providing also initial guidelines on which classifier to choose among five classic classification paradigms.

Machine Learning,Neural and Evolutionary Computing
A Modified Algorithm Based on the Bloom Filter

Mai Jiang,Chunsheng Zhao,Xiang Gao

DOI: https://doi.org/10.1109/cisp.2013.6745220

2013-01-01

Abstract:This paper presents a modified algorithm based on the Bloom Filter. Considering a set A of n elements, if ai is an element of A, then ai could be mapped to a part of the filter by some hash functions. Some eigenvalues are extracted from ai, and then all eigenvalues are combined into a whole: ai*. The ai* is mapped to another part of the filter by some hash functions too. This algorithm can greatly reduce the error rate of the Bloom Filter. The increased cost no matter in computing time and storage space is not high, and realization of the algorithm is easy.
Prefix Filter: Practically and Theoretically Better Than Bloom

Tomer Even,Guy Even,Adam Morrison

DOI: https://doi.org/10.48550/arXiv.2203.17139

2022-03-31

Data Structures and Algorithms

Abstract:Many applications of approximate membership query data structures, or filters, require only an incremental filter that supports insertions but not deletions. However, the design space of incremental filters is missing a "sweet spot" filter that combines space efficiency, fast queries, and fast insertions. Incremental filters, such as the Bloom and blocked Bloom filter, are not space efficient. Dynamic filters (i.e., supporting deletions), such as the cuckoo or vector quotient filter, are space efficient but do not exhibit consistently fast insertions and queries. In this paper, we propose the prefix filter, an incremental filter that addresses the above challenge: (1) its space (in bits) is similar to state-of-the-art dynamic filters; (2) query throughput is high and is comparable to that of the cuckoo filter; and (3) insert throughput is high with overall build times faster than those of the vector quotient filter and cuckoo filter by $1.39\times$-$1.46\times$ and $3.2\times$-$3.5\times$, respectively. We present a rigorous analysis of the prefix filter that holds also for practical set sizes (i.e., $n=2^{25}$). The analysis deals with the probability of failure, false positive rate, and probability that an operation requires accessing more than a single cache line.
FeCBF: A Novel Sub-Optimal Cascaded Bloom Filter Structure Based on Feature Extraction

Quang-Manh Duong,Khoa-Sang Nguyen,Hai-Duong Nguyen,Xuan-Uoc Dao,Anh-Tuan do,Quang-Kien Trinh

DOI: https://doi.org/10.1109/access.2024.3399062

IF: 3.9

2024-05-22

IEEE Access

Abstract:This work presents the Feature extraction Cascaded Bloom Filter (FeCBF), a novel probabilistic data structure formed by cascading multiple Bloom filters in an optimum sequence. The FeCBF's distinctive design of alternating positive and negative filter layers effectively suppresses False Positive/Negative Rates (FPR/FNR), enabling exact filtering with reasonable resource cost. Compared to other state-of-the-art designs on the same experimental dataset, FeCBF demonstrates significant memory space savings of 45% to 76% while maintaining the best FPR in its class. The proposed model also includes a closed-form expression for determining the sub-optimal FeCBF configuration based on desired filter performance metrics, offering the potential for automatic design flow. The FeCBF architecture, designed for hardware implementation, holds promise for many applications. It can be readily deployed as an accelerator in various computing problems, including massive content filtering, network traffic filtering, and online malware/virus detection.

computer science, information systems,telecommunications,engineering, electrical & electronic
Beyond Bloom: A Tutorial on Future Feature-Rich Filters

Prashant Pandey,Martin Farach-Colton,Niv Dayan,Huanchen Zhang

DOI: https://doi.org/10.1145/3626246.3654681

2024-01-01

Abstract:Filters, such as Bloom, quotient, and cuckoo, save space by maintaining an approximate representation of a set and occasionally returning false positives. Filters play a critical role in building modern dataintensive applications and are used across various domains such as databases, storage engines, computational biology, cyber- security, and networks. There has been extensive research on filters in the past few decades resulting in filters with much improved performance and features. Yet modern data-intensive applications are still designed around the limitations of traditional filters resulting in complex designs and sub-optimal performance. This tutorial aims to bring together researchers at the forefront of filter data structure research to help the database community learn about the recent advancements in the theory and practice of filters. The tutorial will cover real-world case studies of redesigning applications using the modern filter APIs to achieve simplicity and improved application performance. The tutorial will further help uncover the open research problems, both in theory and systems, and increase interaction among researchers to tackle those problems.

Optimizing Learned Bloom Filters by Sandwiching

A Model for Learned Bloom Filters, and Optimizing by Sandwiching

A Model for Learned Bloom Filters and Related Structures

A Critical Analysis of Classifier Selection in Learned Bloom Filters

Optimizing Collections of Bloom Filters within a Space Budget

Fast Construction of Partitioned Learned Bloom Filter with Theoretical Guarantees

Stable Learned Bloom Filters for Data Streams

Fast Partitioned Learned Bloom Filter

Optimizing Bloom Filter: Challenges, Solutions, and Comparisons

Adversary Resilient Learned Bloom Filters

A Pareto Optimal Bloom Filter Family with Hash Adaptivity

Retouched Bloom Filters: Allowing Networked Applications to Flexibly Trade Off False Positives Against False Negatives

Bloofi: Multidimensional Bloom filters

Invertible Bloom Lookup Tables

Reducing the number of Bloom filters

On the Evolutionary of Bloom Filter False Positives - An Information Theoretical Approach to Optimizing Bloom Filter Parameters

On the Choice of General Purpose Classifiers in Learned Bloom Filters: An Initial Analysis Within Basic Filters

A Modified Algorithm Based on the Bloom Filter

Prefix Filter: Practically and Theoretically Better Than Bloom

FeCBF: A Novel Sub-Optimal Cascaded Bloom Filter Structure Based on Feature Extraction

Beyond Bloom: A Tutorial on Future Feature-Rich Filters