Compressing Weight-updates for Image Artifacts Removal Neural Networks

Yat Hong Lam,Alireza Zare,Caglar Aytekin,Francesco Cricri,Jani Lainema,Emre Aksu,Miska Hannuksela

DOI: https://doi.org/10.48550/arXiv.1905.04079

2019-06-14

Abstract:In this paper, we present a novel approach for fine-tuning a decoder-side neural network in the context of image compression, such that the weight-updates are better compressible. At encoder side, we fine-tune a pre-trained artifact removal network on target data by using a compression objective applied on the weight-update. In particular, the compression objective encourages weight-updates which are sparse and closer to quantized values. This way, the final weight-update can be compressed more efficiently by pruning and quantization, and can be included into the encoded bitstream together with the image bitstream of a traditional codec. We show that this approach achieves reconstruction quality which is on-par or slightly superior to a traditional codec, at comparable bitrates. To our knowledge, this is the first attempt to combine image compression and neural network's weight update compression.

Machine Learning,Multimedia

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how to effectively remove compression artifacts during the image compression process while maintaining a low bit rate. Specifically, the author proposes a novel method. By optimizing and compressing the weight - updates of the neural network, these updates can be efficiently encoded and included in the bitstream of the traditional codec. This method can not only improve the quality of the decoded image, but also will not significantly increase the total bit rate.

### Main problems:
1. **Removal of compression artifacts**: Traditional lossy compression methods (such as JPEG) introduce compression artifacts (such as blocking artifacts and quantization artifacts) in the decoded image, which seriously affect the perceived quality of the image.
2. **Efficient weight - update compression**: How to efficiently encode and transmit the weight - updates of the neural network used to remove artifacts to the decoding end without significantly increasing the bit rate.

### Solutions:
- **Pre - training and fine - tuning**: First, pre - train the neural network using a large number of images with compression artifacts in an offline phase. Then, in the encoding phase, fine - tune the pre - trained network for specific test images to adapt to specific content.
- **Weight - update compression**: In order to make the weight - updates more compressible, the author introduces an additional loss term, which encourages the weight - updates to become sparse and close to the quantization values. This allows the final weight - updates to be further compressed by pruning and quantization and included in the encoded bitstream at a lower bit rate.
- **Decoding - end application**: At the decoding end, the decoded image is processed using the updated neural network, thereby removing the compression artifacts. After the processing is completed, the network weights are reset to the initial values of the pre - training, so as to prepare for the next batch of images.

### Formula representation:
The weight - update is defined as:
\[
\Delta w = -\rho \nabla_w L(w_t)
\]
where \(\rho\) is the learning rate, \(\nabla\) is the gradient operator, and \(L\) is the loss function.

The cumulative weight - update is defined as:
\[
\Delta w_{\text{acc}} = w_t - w_0
\]
where \(w_0\) is the pre - trained weight.

The compression objective function \(L_{\text{comp}}\) is defined as:
\[
L_{\text{comp}}(\Delta w_{\text{acc}}) = \frac{|\Delta w_{\text{acc}}|}{||\Delta w_{\text{acc}}||} + \alpha ||\Delta w_{\text{acc}}||^2
\]

The total loss function \(L_{\text{total}}\) is defined as:
\[
L_{\text{total}} = L_{\text{mse}} + \gamma L_{\text{comp}}
\]
where \(\gamma\) is a dimensionless parameter used to adjust the influence of the compression objective.

Through this method, the author shows that the quality of the decoded image can be significantly improved without significantly increasing the bit rate.