Abstract:Database-as-a-service offerings enable users to quickly create and deploy complex data processing pipelines. In practice, these pipelines often exhibit significant overlap of computation due to redundant execution of certain sub-queries. It is challenging for developers and database administrators to manually detect overlap across queries since they may be distributed across teams, organization roles, and geographic locations. Thus, we require automated cloud-scale tools for identifying equivalent queries to minimize computation overlap. State-of-the-art algebraic approaches to automated verification of query equivalence suffer from two limitations. First, they are unable to model the semantics of widely-used SQL features, such as complex query predicates and three-valued logic. Second, they have a computationally intensive verification procedure. These limitations restrict their efficacy and efficiency in cloud-scale database-as-a-service offerings. This paper makes the case for an alternate approach to determining query equivalence based on symbolic representation. The key idea is to effectively transform a wide range of SQL queries into first order logic formulae and then use satisfiability modulo theories to efficiently verify their equivalence. We have implemented this symbolic representation-based approach in EQUITAS. Our evaluation shows that EQUITAS proves the semantic equivalence of a larger set of query pairs compared to algebraic approaches and reduces the verification time by 27X. We also demonstrate that on a set of 17,461 real-world SQL queries, it automatically identifies redundant execution across 11% of the queries. Our symbolic-representation based technique is currently deployed on Alibaba's MaxCompute database-as-a-service platform.

PARALLEL VERIFICATION EXECUTION WITH VERIFY ALGEBRA IN A CLOUD ENVIRONMENT

Parallel Verification of Software Architecture Design

Formal Verification Technique for Grid Service Chain Model and Its Application

Automated verification of query equivalence using satisfiability modulo theories

Test Algebra for Combinatorial Testing

Practical Verifiable Computation–A MapReduce Case Study

Using PVS for Modeling and Verifying Cloud Services and Their Composition

Scalable Verification Framework for C Program

Verifying the safety properties of distributed systems via mergeable parallelism

Proof-Carrying Cloud Computation: the Case of Convex Optimization

Parallel Software Model Checking

Distributed Bounded Model Checking

Outsourcing Eigen-Decomposition and Singular Value Decomposition of Large Matrix to a Public Cloud

SMT-Based Modeling and Verification of Cloud Applications.

SLA-Verifier: Stateful and Quantitative Verification for Service Chaining

LFVeri: Network Configuration Verification for Virtual Private Cloud Networks

A Non-Functional Property Based Service Selection and Service Verification Model

Collaborative Verification Scheme for System Functional Properties

Vsfc: Generic and Agile Verification of Service Function Chains in the Cloud

Parallel cloud solution of large algebraic multivalued systems

Verifying Parallel Low-Level Programs for Multi-core Processor