Abstract:The amount of data managed in today's Cloud systems has reached an unprecedented scale. In order to speed up query processing, an effective mechanism is to build indexes on attributes that are used in query predicates. However, conventional indexing schemes fail to provide a scalable service: as the size of these indexes are proportional to the data size, it is not space efficient to build many indexes. As such, it becomes more crucial to develop effective index to provide scalable database services in the Cloud. In this paper, we propose a compact bitmap indexing scheme for a large-scale data store. The bitmap indexing scheme combines state-of-the-art bitmap compression techniques, such as WAH encoding and bit-sliced encoding. To further reduce the index cost, a novel and query efficient partial indexing technique is adopted, which dynamically refreshes the index to handle updates and process queries. The intuition of our indexing approach is to maximize the number of indexed attributes, so that a wider range of queries, including range and join queries, can be efficiently supported. Our indexing scheme is light-weight and its creation can be seamlessly grafted onto the MapReduce processing engine without incurring significant running cost. Moreover, the compactness allows us to maintain the bitmap indexes in memory so that performance overhead of index access is minimal. We implement our indexing scheme on top of the underlying Distributed File System (DFS) and evaluate its performance on an in-house cluster. We compare our index-based query processing with HadoopDB to show its superior performance. Our experimental results confirm the effectiveness, efficiency and scalability of the indexing scheme.

Performance evaluation of air indexing schemes for multi-attribute data broadcast

An Error-Resilient and Tunable Distributed Indexing Scheme for Wireless Data Broadcast

Two-Tier Air Indexing for On-Demand XML Data Broadcast

Energy Efficient Broadcast in Multiradio Multichannel Wireless Networks

Energy-efficient broadcasting algorithm in mobile ad hoc networks

Adaptive data dissemination in wireless sensor networks

A Novel Air Index Scheme for Twig Queries in On-Demand XML Data Broadcast

A Parameterized Flexible Indexing Scheme for Data Broadcast in Wireless Mobile Environments.

Adaptive Data Storage for Querying in Wireless Sensor Networks

An automaton-based index scheme for on-demand XML data broadcast

Energy-efficient shortest path query processing on air.

Multi-Attribute Comprehensive Evaluation Method for Power Transmission and Transformation Projects

Energy-Conserving Fragment Methods for Skewed XML Data Access in Push-Based Broadcast.

On-Demand XML Data Broadcast in Wireless Computing Environments

An Efficient and Compact Indexing Scheme for Large-Scale Data Store.

An Affinity Path Based Wireless Broadcast Scheduling Approach for Multi-item Queries

Efficient Data Scheduling for Multi-item Queries in On-Demand Broadcast

Indexing multi-dimensional data in a cloud system.

Efficient B-tree Based Indexing for Cloud Data Processing.

Efficient Distributed Query Processing in Large RFID-enabled Supply Chains

Indexing Linked Data in a Wireless Broadcast System with 3D Hilbert Space-Filling Curves