Adaptive Segmentation-Based Initialization for Steered Mixture of Experts Image Regression

Yi-Hsin Li,Sebastian Knorr,Mårten Sjöström,Thomas Sikora
2024-09-16
Abstract:Kernel image regression methods have shown to provide excellent efficiency in many image processing task, such as image and light-field compression, Gaussian Splatting, denoising and super-resolution. The estimation of parameters for these methods frequently employ gradient descent iterative optimization, which poses significant computational burden for many applications. In this paper, we introduce a novel adaptive segmentation-based initialization method targeted for optimizing Steered-Mixture-of Experts (SMoE) gating networks and Radial-Basis-Function (RBF) networks with steering kernels. The novel initialization method allocates kernels into pre-calculated image segments. The optimal number of kernels, kernel positions, and steering parameters are derived per segment in an iterative optimization and kernel sparsification procedure. The kernel information from "local" segments is then transferred into a "global" initialization, ready for use in iterative optimization of SMoE, RBF, and related kernel image regression methods. Results show that drastic objective and subjective quality improvements are achievable compared to widely used regular grid initialization, "state-of-the-art" K-Means initialization and previously introduced segmentation-based initialization methods, while also drastically improving the sparsity of the regression models. For same quality, the novel initialization results in models with around 50% reduction of kernels. In addition, a significant reduction of convergence time is achieved, with overall run-time savings of up to 50%. The segmentation-based initialization strategy itself admits heavy parallel computation; in theory, it may be divided into as many tasks as there are segments in the images. By accessing only four parallel GPUs, run-time savings of already 50% for initialization are achievable.
Computer Vision and Pattern Recognition
What problem does this paper attempt to address?
The paper attempts to address the issue of high computational burden in the optimization process of the Steered Mixture of Experts (SMoE) model. Specifically: - **High computational burden**: Currently, when optimizing SMoE network parameters, especially using iterative strategies (such as expectation maximization or gradient descent) for parameter estimation, parameter initialization is crucial for algorithm convergence speed, reconstruction quality, and runtime. However, existing initialization methods are insufficiently explored for the SMoE model. - **Proposed new method**: The paper introduces a novel adaptive segmentation initialization strategy aimed at improving the performance of the SMoE model and related kernel image regression methods by locally allocating kernels. This method improves the existing technology in the following three aspects: - **Capturing high-frequency details**: By using image segmentation to allocate an adaptive number of kernels to the image, it better captures high-frequency details. - **Reducing optimization time**: Due to well-initialized kernels, the optimization time is significantly shortened, ensuring that the initial kernels are closer to the final optimized results, thereby reducing the time required to reach the final optimization. - **Wide applicability**: Although designed specifically for the SMoE model, this method can also be used to initialize other kernel regression models, such as radial basis function (RBF) networks. By introducing this new initialization method, the paper aims to improve the efficiency and performance of the SMoE model and its related kernel image regression methods, especially when dealing with complex image features and large-scale datasets.