Abstract:Image classification is a fundamental building block for a majority of computer vision applications. With the growing popularity and capacity of machine learning models, people can easily access trained image classifiers as a service online or offline. However, model use comes with a cost and classifiers of higher capacity usually incur higher inference costs. To harness the respective strengths of different classifiers, we propose a principled approach, OCCAM, to compute the best classifier assignment strategy over image classification queries (termed as the optimal model portfolio) so that the aggregated accuracy is maximized, under user-specified cost budgets. Our approach uses an unbiased and low-variance accuracy estimator and effectively computes the optimal solution by solving an integer linear programming problem. On a variety of real-world datasets, OCCAM achieves 40% cost reduction with little to no accuracy drop.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to maximize the aggregate accuracy of classifiers in the image classification task under the premise of meeting the cost budget specified by the user. Specifically, the paper proposes a method named OCCAM (Optimization with Cost Constraints for Accuracy Maximization), aiming to optimize the processing strategy of image classification queries by reasonably allocating classifiers with different capacities and costs. ### Problem Background With the popularization and development of machine - learning models, people can easily access trained image classifiers online or offline. However, using these models has costs, especially high - performance classifiers usually bring higher inference costs. In order to make full use of the advantages of different types of classifiers, the paper proposes a principled method - OCCAM to calculate the optimal classifier allocation strategy (i.e., the optimal model combination) so as to maximize the aggregate accuracy within a given cost budget. ### Main Challenges 1. **Trade - off between cost and accuracy**: Although large - scale classifiers have high accuracy, their hardware usage and inference time costs are also high. Small - scale classifiers have low costs, but their accuracy may be insufficient in some cases. 2. **Dynamic adjustment**: Users hope to be able to flexibly adjust the selection of classifiers under different cost budgets to achieve the best balance between cost and accuracy. ### OCCAM's Solutions OCCAM solves the above problems through the following steps: 1. **Define the optimal model combination problem**: Formally define the problem of how to allocate classifiers to maximize the aggregate accuracy under the given cost budget and a set of classifiers with different capacities and costs. 2. **Unbiased low - variance accuracy estimator**: Propose an unbiased low - variance accuracy estimator based on statistical guarantees to evaluate the test accuracy of each classifier on a given query. 3. **Integer linear programming (ILP)**: Model the optimal model combination problem as an integer linear programming problem and use a standard ILP solver (such as HiGHS) to solve it to find the optimal allocation strategy under the given cost budget. ### Experimental Results The paper conducts experiments on multiple real - world datasets, and the results show that OCCAM can significantly reduce the inference cost with almost no loss in accuracy. For example, on the Tiny ImageNet dataset, OCCAM achieves about 20% cost reduction while the accuracy drops by less than 1%. ### Formula Summary - **Success probability function**: \[ SP_i(x)=f_i(x)[O(x)] = \Pr[f_i(x)=O(x)] \] where \(SP_i(x)\) represents the probability that the classifier \(f_i\) correctly predicts the query \(x\). - **Expected accuracy**: \[ E[\text{Accuracy}(\mu)]=\frac{1}{|X|}\sum_{x\in X}SP_{\mu(x)}(x) \] - **Integer linear programming problem**: \[ \max\sum_{i = 1}^{M}\sum_{j = 1}^{N}dSP_i(x_j)\cdot t_{i,j} \] Constraints: \[ \sum_{i = 1}^{M}\sum_{j = 1}^{N}b_i\cdot t_{i,j}\leq B \] \[ \sum_{i = 1}^{M}t_{i,j}=1\quad\text{for }j = 1,2,\dots,N \] \[ t_{i,j}\in\{0,1\} \] Through these methods, OCCAM successfully solves the problem of how to maximize the classification accuracy under a limited cost budget in the image classification task.

OCCAM: Towards Cost-Efficient and Accuracy-Aware Image Classification Inference

Efficient IoT Inference via Context-Awareness

Multimodal Adaptive Inference for Document Image Classification with Anytime Early Exiting

Cost-effective land cover classification for remote sensing images

Problem-dependent attention and effort in neural networks with applications to image resolution and model selection

OCCAM: An Optimization-Based Approach to Network Inference

Saliency-Aware Image-To-Class Distances for Image Classification

Simple Image-level Classification Improves Open-vocabulary Object Detection

Online Algorithms for Hierarchical Inference in Deep Learning applications at the Edge

Adaptive learning cost‐sensitive convolutional neural network

Cost-Effective Active Learning for Deep Image Classification

Anytime Induction of Low-cost, Low-error Classifiers: a Sampling-based Approach

LCA-on-the-Line: Benchmarking Out-of-Distribution Generalization with Class Taxonomies

Optimizer's Information Criterion: Dissecting and Correcting Bias in Data-Driven Optimization

One-Class Classification: A Survey

Improved Decision Module Selection for Hierarchical Inference in Resource-Constrained Edge Devices

Context-aware Optimization for Bandwidth-Efficient Image Analytics Offloading

Learning with cost intervals.

Cost-sensitive matrixized classification learning with information entropy

CESED: Exploiting Hyperspherical Predefined Evenly-Distributed Class Centroids for OOD Detection.

IMMC: Incremental Maximum Margin Criterion.