Abstract:Diffusion models, known for their generative capabilities, have recently shown unexpected potential in image classification tasks by using Bayes' theorem. However, most diffusion classifiers require evaluating all class labels for a single classification, leading to significant computational costs that can hinder their application in large-scale scenarios. To address this, we present a Hierarchical Diffusion Classifier (HDC) that exploits the inherent hierarchical label structure of a dataset. By progressively pruning irrelevant high-level categories and refining predictions only within relevant subcategories, i.e., leaf nodes, HDC reduces the total number of class evaluations. As a result, HDC can accelerate inference by up to 60% while maintaining and, in some cases, improving classification accuracy. Our work enables a new control mechanism of the trade-off between speed and precision, making diffusion-based classification more viable for real-world applications, particularly in large-scale image classification tasks.

What problem does this paper attempt to address?

### What problem does this paper attempt to solve? This paper aims to solve the computational efficiency problem of diffusion models in image classification tasks. Specifically, traditional Diffusion Classifiers need to evaluate all class labels when performing a single classification, which leads to significant computational costs and limits their application in large - scale scenarios. To solve this problem, the authors propose a new method - **Hierarchical Diffusion Classifier (HDC)**. This method utilizes the inherent hierarchical label structure of the dataset, gradually pruning irrelevant high - level categories and refining predictions only in relevant sub - categories, thereby reducing the total number of category evaluations. HDC can accelerate the inference process while maintaining or even improving classification accuracy, with a speed - up of up to 60%. ### Main contributions 1. **Accelerate the inference time of zero - sample diffusion classifiers**: Significantly reduce the computational complexity through hierarchical label search while maintaining similar accuracy. 2. **Propose a novel Hierarchical Diffusion Classifier (HDC)**: Efficiently narrow the range of candidate categories by using the dataset label structure of ImageNet - 1K. 3. **Achieve faster inference time**: Compared with traditional diffusion classifiers, the HDC framework achieves an acceleration of approximately 60% while maintaining or improving classification performance (for example, from 64.90% to 65.16%). ### Method overview The core idea of HDC is to refine high - level categories (such as "animals" or "tools") into more specific and actual categories (such as "hammerhead sharks" or "screwdrivers") by hierarchically evaluating labels and gradually pruning irrelevant categories. This method can significantly reduce the number of categories that need to be evaluated, thereby speeding up prediction and potentially improving accuracy. The specific steps are as follows: 1. **Initialization**: Start from the root node, which contains the highest - level categories. 2. **Error scoring**: Calculate the error score of each sub - node using Monte Carlo estimation. 3. **Pruning**: According to the set pruning strategy (such as fixed - ratio or dynamic pruning), select the node with the smallest error to enter the next layer. 4. **Final prediction**: When reaching the leaf node, use the classic diffusion classifier for final classification. In this way, HDC can significantly reduce the demand for computational resources while maintaining classification accuracy, making it more suitable for large - scale image classification tasks.

Just Leaf It: Accelerating Diffusion Classifiers with Hierarchical Class Pruning

Nested Diffusion Models Using Hierarchical Latent Priors

Hierarchically branched diffusion models leverage dataset structure for class-conditional generation

Hierarchical Clustering for Conditional Diffusion in Image Generation

Effortless Efficiency: Low-Cost Pruning of Diffusion Models

A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Data

Enhancing Diffusion-Based Image Synthesis with Robust Classifier Guidance

Pruning then Reweighting: Towards Data-Efficient Training of Diffusion Models

Cost-sensitive Learning of Hierarchical Tree Classifiers for Large-Scale Image Classification and Novel Category Detection

Efficient discriminative learning of class hierarchy for many class prediction

Attention Tree: Learning Hierarchies of Visual Features for Large-Scale Image Recognition

DeepCache: Accelerating Diffusion Models for Free

Efficient Image-to-Image Diffusion Classifier for Adversarial Robustness

Robust Classification via a Single Diffusion Model

An Efficient Framework for Enhancing Discriminative Models via Diffusion Techniques

Improving Efficiency of Diffusion Models via Multi-Stage Framework and Tailored Multi-Decoder Architectures

Probabilistic Cascading for Large Scale Hierarchical Classification

Tracing the Roots: Leveraging Temporal Dynamics in Diffusion Trajectories for Origin Attribution

Relational Diffusion Distillation for Efficient Image Generation

Deep Multi-task Learning for Hierarchical Classification

FineDiffusion: Scaling up Diffusion Models for Fine-grained Image Generation with 10,000 Classes