Abstract:Machine Learning (ML) applications are proliferating in the enterprise. Relational data which are prevalent in enterprise applications are typically normalized; as a result, data has to be denormalized via primary/foreign-key joins to be provided as input to ML algorithms. In this paper, we study the implementation of popular nonlinear ML models, Gaussian Mixture Models (GMM) and Neural Networks (NN), over normalized data addressing both cases of binary and multiway joins over normalized relations. For the case of GMM, we show how it is possible to decompose computation in a systematic way both for binary joins and for multi-way joins to construct mixture models. We demonstrate that by factoring the computation, one can conduct the training of the models much faster compared to other applicable approaches, without any loss in accuracy. For the case of NN, we propose algorithms to train the network taking normalized data as the input. Similarly, we present algorithms that can conduct the training of the network in a factorized way and offer performance advantages. The redundancy introduced by denormalization can be exploited for certain types of activation functions. However, we demonstrate that attempting to explore this redundancy is helpful up to a certain point; exploring redundancy at higher layers of the network will always result in increased costs and is not recommended. We present the results of a thorough experimental evaluation, varying several parameters of the input relations involved and demonstrate that our proposals for the training of GMM and NN yield drastic performance improvements typically starting at 100%, which become increasingly higher as parameters of the underlying data vary, without any loss in accuracy.

Efficient normalization techniques to optimize AI models for deployment in tactical edge

Class-Aware Pruning for Efficient Neural Networks

Achieving Pareto Optimality using Efficient Parameter Reduction for DNNs in Resource-Constrained Edge Environment

Unified Normalization for Accelerating and Stabilizing Transformers

A Survey on Optimization Techniques for Edge Artificial Intelligence (AI)

Edge AI: Evaluation of Model Compression Techniques for Convolutional Neural Networks

Multi-Component Optimization and Efficient Deployment of Neural-Networks on Resource-Constrained IoT Hardware

AntiDote: Attention-based Dynamic Optimization for Neural Network Runtime Efficiency

Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Training

Detecting Dead Weights and Units in Neural Networks

Maintaining Performance with Less Data

Enhancing Reliability of Neural Networks at the Edge: Inverted Normalization with Stochastic Affine Transformations

Optimized Transformer Models: ℓ′ BERT with CNN-like Pruning and Quantization

Normalization Techniques in Training DNNs: Methodology, Analysis and Application

Batchless Normalization: How to Normalize Activations Across Instances with Minimal Memory Requirements

New Interpretations of Normalization Methods in Deep Learning.

Impact of ML Optimization Tactics on Greener Pre-Trained ML Models

Structured Model Pruning for Efficient Inference in Computational Pathology

Small Contributions, Small Networks: Efficient Neural Network Pruning Based on Relative Importance

UPSCALE: Unconstrained Channel Pruning

Efficient Construction of Nonlinear Models over Normalized Data