Description of video coding technology proposal by Microsoft

Feng Wu,Xiaoyan Sun,Jizheng Xu,You Zhou,Wenpeng Ding,Xiulian Peng,Zhiwei Xiong

2010-01-01

Abstract:This document describes the details of the response from Microsoft to the Call for Proposal of new video coding technologies issued jointly by ISO/IEC and ITU-T. Additionally, an excel file is attached to present the per-frame PSNRs of the decoded video.

What problem does this paper attempt to address?

A Video Coding Algorithm Using Vector-Based Techniques.

WP Li,HQ Cao,SP Li,F Ling,SA Segan,HQ Sun,JP Wus,YQ Zhang

DOI: https://doi.org/10.1109/76.554426

IF: 5.859

1997-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:This paper presents an algorithm proposal submitted to MPEG-4 for video coding. The proposed algorithm addresses the functionality of improved coding efficiency for compression. It uses vector-based techniques for coding intraframes (the first frame and subsequent refreshing key frames) and motion-compensated difference frames. It uses the same motion estimation and motion compensation techniques as H.263. A video frame (I or P frame) is first decomposed into a set of vector bands using a vector wavelet transform. This stage of vector-based signal processing makes subsequent vector quantization in the vector bands very efficient. Lattice vector quantization is then used in the vector bands. A 100% labeling efficiency is achieved for lattice vector quantization by using a set of generalized labeling algorithms for various important lattices with pyramid and sphere boundaries. Finally, entropy coding is used to code the indexes generated from lattice vector quantization. Our coding results have shown that a gain in peak signal-to-noise ratio (PSNR) up to 8 dB for intraframe coding and up to 6 dB for interframe coding can be achieved over H.263. Subjective quality improvement of the proposed algorithm over H.263 can be easily observed.
Multi-View Video Coding Based on Vector Estimation and Weighted Disparity Interpolation

Suxing Liu,Ping An,Zhaoyang Zhang,Qian Zhang,Tao Yan

DOI: https://doi.org/10.1007/s00034-009-9129-0

2009-01-01

Abstract:Aiming at fully exploiting the temporal and spatial redundancy and answering the “Call for Proposals” for multi-view video coding (MVC) issued by MPEG, a MVC scheme based on vector field estimation and weighted disparity interpolation is presented. By extending the loop constraint to multi-view images for a parallel camera model and proposing the novel “vector field estimation” scheme, the temporal and spatial redundancy is significantly reduced. Also, weighted disparity interpolation is performed to predict adjacent disparity vectors. Experimental results over multi-view image sets imply that the coding efficiency is improved about 0.2–0.5 dB compared with previous coding approaches such as H.264/AVC simulcast and JMVM.
A Video Coding System with Spatial-Temporal Down-/Up-Sampling and Super-Resolution Reconstruction

Xiaohai He,Wenjie Fu,Hongwei Lin,Xueqing Li,Xin Peng

DOI: https://doi.org/10.1109/cicn.2016.53

2016-01-01

Abstract:A video coding system based on spatial-temporal down-sampling/up-sampling and super-resolution reconstruction method is proposed. The method is an exploration and providing ideas for our future work. The system pre-reduces the spatial resolutions and frame rates of source videos before being encoded by a standardized codec such as HEVC/H.265. On the reception side, the decoded video is up-sampled to four times resolution, and the frame rate is also increased to original frame rate. We tested the performance in combination with a 2K 60P HEVC/H.265 encoder. The experiment results are shown in this paper.
Video Codec Using Flexible Block Partitioning and Advanced Prediction, Transform and Loop Filtering Technologies

K. Choi,J. Chen,M. W. Park,H. Yang,W. Choi,S. Ikonin,Y. Piao,S. Esenlik,M. Park,Y.-K. Wang,N. Choi,Y. Zhao,S. Jeong,H. Chen,A. Tamse,A. Filippov,X. Ma,J. Min,R. Chernyak,B. Jin,A. M. Kotra,S. Lee,H. Gao,C. Kim,T. Solovyev,K. P. Choi,V. Rufitskiy,M. Sychev,W. Xu,J. Park,T Wang

DOI: https://doi.org/10.1109/TCSVT.2020.2971268

IF: 5.859

2020-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:This paper describes a joint response to the Call for Proposals by Samsung, Huawei, GoPro, and HiSilicon on Video Compression with Capability beyond HEVC/H.265, jointly issued by ITU-T SG16 Q.6 (VCEG) and ISO/IEC JTC1/SC29/WG11 (MPEG). In the proposed codec, the coding framework supports hierarchical splitting with binary and ternary trees and flexible coding order representations. Additionally, n...
Extended application of scalable video coding methods

Zhi-gang Li,Zhao-yang Zhang,Biao Wu,Ying Zhang

DOI: https://doi.org/10.1007/11559573_45

2005-01-01

Abstract:SP(Synchronization-Predictive) frame coding, which enables high efficiency of switching between two video bitstreams with different qualities, is supported by H.264/AVC. And FGS(Fine-Granular-Scalability) coding is supported by MPEG-4 video standard. This paper proposes a solution for combination these two tools with each other so as to adapt to high bandwidth variations of Internet or wireless networks and to low bandwidth variations flexibly for transmitted video streams. Experimental results show that our proposed system outperforms FGS coding by 0.47dB and the H.264/AVC-based video stream switching approach by 0.23dB on average.
A Network-Friendly Architecture for Multi-View Video Coding (Mvc)

Liang-Hao Wang,Ming Xi,Dong-Xiao Li,Ming Zhang

DOI: https://doi.org/10.4028/www.scientific.net/amr.121-122.678

2010-01-01

Advanced Materials Research

Abstract:Multi-view Video Coding (MVC) is very promising in applicable field for its 3D effect and interactive functions (multi viewpoint). In this paper, a network-friendly architecture for MVC is proposed. To exploit temporal as well as inter-view dependencies between adjacent cameras, two main features of the coder are used: hierarchical B picture and FGS (fine granularity scalable). Coding results are shown for the proposed multi-view coder and compared to the traditional coding architectures to show that our presented coding scheme outperforms the other approaches for the tested sequence.
An efficient fine grained multi-description and scalable video coding scheme

Anbang Zhao,Wensheng Wang,Huijuan Cui,Kun Tang

2006-01-01

Abstract:In this paper we propose a simple and efficient Multiple Description Coding (MDC) algorithm based on weighted signal combination. The proposed scheme adopts the standard video encoder by incorporating a pre-processing stage to generate multi-descriptions, on the other hand the decoder employs a post-processing algorithm to combine the descriptions to provide better image quality. Furthermore, a scalable codec is introduced to the MDC system which allows the system to provide multi-description and scalable capability at the same time. In addition, since the different scalable descriptions may have different qualities, the simple averaging process is not optimal. The last but not the least, the optimal weighted combination scheme of the two descriptions is derived based on their respective signal to noise ratio. Compared with the simple average combination, the proposed scheme can improve the video quality significantly, especially when the quality difference between the two descriptions becomes larger, the gain can be up to 2.38dB and 3.56dB.
A novel video coding framework by perceptual representation and macroblock-based matching pursuit algorithm

Jianning Zhang,Lifeng Sun,Yuzhuo Zhong

DOI: https://doi.org/10.1007/978-3-540-69423-6_32

2007-01-01

Abstract:This paper presents a novel hybrid video coding framework by perceptual representation and macroblock-based matching pursuit algorithm (PRMBMP), which uses a set of filters to extract the perceptual parts of each video frame. The original video frame is separated into low-frequency image and high-frequency image. The low-frequency image has low sensitivity to human perception and few complex texture details, which can be handled efficiently by traditional H.264 video coding. The high-frequency image is the perceptual representation, which includes more texture details and edges. The proposed macroblock-based matching pursuit algorithm is used to compress the high-frequency image, which speeds up the conventional matching pursuit algorithm efficiently by estimating the local optimization. The experiments show that the proposed framework can achieve 2 to 3 dB improvements compared with the conventional H.264 video coding framework. The proposed framework also has the definition scalability, which can be widely used in bandwidth-variation video applications.
Overview of MPEG Internet Video Coding

R. G. Wang,G. Li,S. Park,J. Kim,T. Huang,E. S. Jang,W. Gao

DOI: https://doi.org/10.1117/12.2194387

2015-01-01

Abstract:MPEG has produced standards that have provided the industry with the best video compression technologies. In order to address the diversified needs of the Internet, MPEG issued the Call for Proposals (CfP) for internet video coding in July, 2011. It is anticipated that any patent declaration associated with the Baseline Profile of this standard will indicate that the patent owner is prepared to grant a free of charge license to an unrestricted number of applicants on a worldwide, non-discriminatory basis and under other reasonable terms and conditions to make, use, and sell implementations of the Baseline Profile of this standard in accordance with the ITU-T/ITU-R/ISO/IEC Common Patent Policy. Three different codecs had responded to the CfP, which are WVC, VCB and IVC. WVC was proposed jointly by Apple, Cisco, Fraunhofer HHI, Magnum Semiconductor, Polycom and RIM etc. it's in fact AVC baseline. VCB was proposed by Google, and it's in fact VP8. IVC was proposed by several Universities (Peking University, Tsinghua University, Zhejiang University, Hanyang University and Korea Aerospace University etc.) and its coding tools was developed from Zero. In this paper, we give an overview of the coding tools in IVC, and evaluate its performance by comparing it with WVC, VCB and AVC High Profile.
Multi-view Video Coding Based on View Prediction

Ping An,Qiuyan Guo,Tao Mi,Li Zhou,Zhaoyang Zhang

DOI: https://doi.org/10.1109/icalip.2008.4590138

2008-01-01

Abstract:Multiple view video coding (MVC) is key technology that serves a wide variety of applications, including FTV (free-viewpoint television), 3DTV (three dimensional television) and surveillance. Multi-view video consists of multiple views of the same scene in which there is a high degree of correlation between inter views and intra views. In this paper, a multi-view video coding method based on a new view prediction structure is proposed which makes full use of the spatial redundancy in addition to exploiting the temporal redundancy by performing spatial prediction across different views. A multi-reference disparity prediction method based on variable size block matching is used to get better disparity compensation effect in the proposed structure. Besides, by omitting transmission some B frames, the coding bit-rate is further reduced. Experimental results show that compared to general GoGOP algorithm, the proposed multi-reference prediction algorithm can reconstruct the target images with higher quality, and the MVC scheme based on the new view prediction structure can obtain a coding gain with about 1.4 dB.
Efficient Video Coding Using Legacy Algorithmic Approaches

Jianwen Chen,Feng Xu,Yun He,John Villasenor,Yuxing Han,Yan Xu,Yaocheng Rong,Cliff Reader,Jiangtao Wen

DOI: https://doi.org/10.1109/tmm.2011.2169046

IF: 7.3

2012-01-01

IEEE Transactions on Multimedia

Abstract:We show that for high bit rates, a video coding algorithm using a suitable combination of the QM coder and on other methods first published over 20 years ago can deliver video quality rivaling that of H.264 at lower complexity. This has implications both technically, since encoders built using these methods can be more power efficient, and commercially, given the complex licensing and intellectual property issues that accompany newer coding methods such as H.264 and MPEG-4. The methods described in this paper are the basis for the recent decision of the MPEG standards group to begin work on what is referred to as the “Type-1 Video Coding” standard, which, in addition to aiming for high coding efficiency, is intended to minimize royalty issues.
Highly efficient video compression using quadtree structures and improved techniques for motion representation and entropy coding

H. Schwarz,S. Oudin,H. Kirchhoffer,Tobias Hinz,H. Lakshman,B. Bross,Mischa Siekmann,S. Bosse,Philipp Helle,Karsten Sühring,Martin Winken,Tung Nguyen,D. Marpe,T. Wiegand

DOI: https://doi.org/10.1109/PCS.2010.5702464

2010-12-01

Abstract:This paper describes a novel video coding scheme that can be considered as a generalization of the block-based hybrid video coding approach of H.264/AVC. While the individual building blocks of our approach are kept simple similarly as in H.264/AVC, the flexibility of the block partitioning for prediction and transform coding has been substantially increased. This is achieved by the use of nested and pre-configurable quadtree structures, such that the block partitioning for temporal and spatial prediction as well as the space-frequency resolution of the corresponding prediction residual can be adapted to the given video signal in a highly flexible way. In addition, techniques for an improved motion representation as well as a novel entropy coding concept are included. The presented video codec was submitted to a Call for Proposals of ITU-T VCEG and ISO/IEC MPEG and was ranked among the five best performing proposals, both in terms of subjective and objective quality.

Computer Science
MPEG Internet Video Coding Standard and Its Performance Evaluation

Ronggang Wang,Zhenyu Wang,Kui Fan,Tiejun Huang,Wenmin Wang,Ge Li,Wen Gao

DOI: https://doi.org/10.1109/tcsvt.2016.2631249

IF: 5.859

2016-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:MPEG has produced standards that have provided the industry with the best video compression technologies. To address diverse Internet needs, MPEG issued a Call for Proposals (CfP) for Internet video coding (IVC) in July, 2011. The anticipation is that any patent declaration associated with the baseline profile of this standard will indicate that the patent owner is prepared to grant a free of charge license to an unrestricted number of applicants worldwide. Three codecs have responded to the CfP: Web video coding (WVC), video coding for browsers (VCB), and IVC. WVC is in fact the AVC baseline, and VCB uses the same coding tools as VP8. IVC has been developed in MPEG from scratch by combining well-known existing technology elements and new coding tools with royalty-free declarations. In June 2015, the IVC project was approved as ISO/IEC 14496-33 (MPEG-4 IVC). This standard can be highly beneficial for video services in the Internet domain. This paper describes the main coding tools used in IVC, and evaluates its objective and subjective performances compared with WVC, VCB, and AVC high profile (AVC HP). The experimental results show that IVC's compression performance is approximately equal to that of the AVC HP for typical operational settings, both for streaming and low-delay applications, and is superior to WVC and VCB.
Efficient and universal scalable video coding

Feng Wu,Shipeng Li,Rong Yan,Xiaoyan Sun,Ya-qin Zhang

DOI: https://doi.org/10.1109/ICIP.2002.1039881

2002-01-01

Abstract:This paper proposes a unified efficient and universal scalable video coding framework that supports different scalabilities, such as fine granularity quality, temporal, spatial and complexity scalabilities. The proposed framework is established upon the recent studies in fine granularity scalable (FGS) video coding. It contains two key points. Firstly, in order to improve the coding efficiency of the proposed framework, more than one motion compensation loop is used. Since high quality references are introduced into the enhancement layer coding, the proposed framework can efficiently compress different-resolution video at different layers for the purpose of the complexity and spatial scalability. Secondly, the drifting reduction techniques are studied in this paper. This helps the proposed framework to maintain good performance at lower enhancement bit rates. By defining coding modes, a macroblock level control mechanism is developed to achieve a better trade-off between low drifting errors and high coding efficiency.
Multiple Description Video Coding Using Adaptive Temporal Sub-Sampling

Huihui Bai,Yao Zhao,Ce Zhu

DOI: https://doi.org/10.1109/icme.2007.4284904

2007-01-01

Abstract:Multiple description coding (MDC) is a promising alternative for robust transmission of information over non-prioritized and unpredictable networks. Especially, MDC has emerged as an attractive approach for video applications where retransmission is unacceptable or infeasible. In this paper, an effective MD video codec is designed based on pre-and post-processing of video sequences, without any modification to the source or channel codec. Considering different motion information of inter-frame, adaptive temporal sub-sampling is employed to the original video data. As a result, adaptive redundancy can make a better tradeoff between the reconstruction quality and compression efficiency. The experimental results exhibit better performance of the proposed scheme than other schemes.
Multiple Description Video Coding Using Joint Frame Duplication/Interpolation

Huihui Bai,Yao Zhao,Ce Zhu,Anhong Wang

2012-01-01

Abstract:Multiple description coding (MDC) is a promising alternative to combatting information loss without any retransmission. In this paper, an effective MD video codec is designed based on temporal pre- and post-processing of video sequences without modifying the actual coding process itself, which makes it compatible with the current standard source or channel codec. For ease of post-processing, motion-compensated interpolation (MCI) based on piecewise uniform motion assumption is adopted to estimate the lost frame in side decoding. Accordingly, to match the post-processing, in the pre-processing joint frame duplication/interpolation is first applied to the original video data before performing odd/even frame splitting, which attempts to make the motion variety in the generated descriptions piecewise uniformly thus achieving better side reconstructed quality based on MCI. The experimental results exhibit better performance of the proposed scheme than some other tested schemes, in both the on-off channel environment and packet loss network.
Technical description of the EPFL submission to the JPEG DNA CfP

Davi Lazzarotto,Jorge Encinas Ramos,Michela Testolina,Touradj Ebrahimi

2023-12-01

Abstract:This document provides a technical description of the codec proposed by EPFL to the JPEG DNA Call for Proposals. The codec we refer to as V-DNA for its versatility, enables the encoding of raw images and already compressed JPEG 1 bitstreams, but the underlying algorithm could be used to encode and transcode any kind of data. The codec is composed of two main modules: the image compression module, handled by the state-of-the-art JPEG XL codec, and the DNA encoding module, implemented using a modified Raptor Code implementation following the RU10 (Raptor Unsystematic) description. The code for encoding and decoding, as well as the objective metrics results, plots and biochemical constraints analysis are available on ISO Documents system with document number WG1M101013-ICQ-EPFL submission to the JPEG DNA CfP.

Image and Video Processing
Witsenhausen-Wyner Video Coding

Mei Guo,Zixiang Xiong,Feng Wu,Debin Zhao,Xiangyang Ji,Wen Gao

DOI: https://doi.org/10.1109/tcsvt.2011.2129910

IF: 5.859

2011-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Inspired by Witsenhausen and Wyner's 1980 (now expired) patent on “interframe coder for video signals,” this paper presents a Witsenhausen-Wyner video codec, where the motion-compensated previously decoded video frame is used at the decoder as side information for joint decoding. Specifically, we replace predictive Inter coding in H.264/AVC by the syndrome-based coding scheme of Witsenhausen and Wyner, while keeping the Intra and Skip modes of H.264/AVC unchanged. We employ forward motion estimation at the encoder and send the motion vectors to help generate side information at the decoder, since our focus is not on low-complexity encoding. We also examine the tradeoff between the motion vector resolution and coding efficiency. Within the Witsenhausen-Wyner coding mode, we optimize the decision between syndrome coding and entropy coding among different discrete cosine transform (DCT) bands and among different bit-planes within each DCT coefficient. Extensive simulations of video transmission over wireless networks show that Witsenhausen-Wyner video coding is more robust against channel errors than H.264/AVC. The price paid for enhanced error-resilience with Witsenhausen-Wyner coding is a small loss in compression efficiency.
Brief analysis of the H.264 coding standard

Wu Yun,Zhao Yong,Li Jianshi

DOI: https://doi.org/10.1109/iihmsp.2007.4457675

2007-01-01

Abstract:H.264/AVC is the current video standardization project of the ITU-T Video Coding Experts Group (VCEG) and the ISO/IEC Moving Picture Experts Group (MPEG). H.264/AVC represents a major step forward in the development of video coding standards. In this paper, the authors briefly introduced a technical overview of H.264/AVC standard and proposed the related coding tools. At the last section, the authors compared the H.264/AVC standard with other prior coding standards. The results show that H.264/AVC standard will enable new applications and business opportunities to be developed.
Colour Correction Pre-Processing and Chrominance Reconstruction Post-Processing for Multi-View Video Coding

F. Shao,M. Yu,G. Jiang,K. Chen,Z. Peng

DOI: https://doi.org/10.1049/iet-ipr.2011.0141

IF: 2.3

2012-01-01

IET Image Processing

Abstract:In a multi-view video system, a great amount of data for storage and transmission raise an inevitable problem to be solved. In this study, a new multi-view video coding (MVC) method is proposed by performing colour correction as a preprocessing step and chrominance reconstruction as a post-processing step. In the proposed method, colour correction is first performed to eliminate colour differences among views and to improve coding efficiency simultaneously. Then, chrominance information in some particular views is discarded to reduce bit-rates. Finally, the discarded chrominance information is reconstructed accurately at the decoder. Experimental results show that the proposed method can better increase the bit-rate saving than the original MVC system. Moreover, the proposed method can accomplish higher quality reconstruction at the trivial cost of quality degradation.

Description of video coding technology proposal by Microsoft

A Video Coding Algorithm Using Vector-Based Techniques.

Multi-View Video Coding Based on Vector Estimation and Weighted Disparity Interpolation

A Video Coding System with Spatial-Temporal Down-/Up-Sampling and Super-Resolution Reconstruction

Video Codec Using Flexible Block Partitioning and Advanced Prediction, Transform and Loop Filtering Technologies

Extended application of scalable video coding methods

A Network-Friendly Architecture for Multi-View Video Coding (Mvc)

An efficient fine grained multi-description and scalable video coding scheme

A novel video coding framework by perceptual representation and macroblock-based matching pursuit algorithm

Overview of MPEG Internet Video Coding

Multi-view Video Coding Based on View Prediction

Efficient Video Coding Using Legacy Algorithmic Approaches

Highly efficient video compression using quadtree structures and improved techniques for motion representation and entropy coding

MPEG Internet Video Coding Standard and Its Performance Evaluation

Efficient and universal scalable video coding

Multiple Description Video Coding Using Adaptive Temporal Sub-Sampling

Multiple Description Video Coding Using Joint Frame Duplication/Interpolation

Technical description of the EPFL submission to the JPEG DNA CfP

Witsenhausen-Wyner Video Coding

Brief analysis of the H.264 coding standard

Colour Correction Pre-Processing and Chrominance Reconstruction Post-Processing for Multi-View Video Coding