Guest Editorial: Content-Aware Visual Systems: Analysis, Streaming, and Retargeting

Gao, W.,Chang Wen Chen,Ariel Shamir,Yan, B.

DOI: https://doi.org/10.1109/JETCAS.2014.2300571

2014-01-01

Abstract:The 13 papers in this special issue can be classified into five broad thematic categories: representation, streaming, retargeting, 3-D video, and system.

What problem does this paper attempt to address?

Guest Editorial Introduction to the Special Section on Intelligent Visual Content Analysis and Understanding

Hongliang Li,Lu Fang,Tianzhu Zhang

DOI: https://doi.org/10.1109/tcsvt.2020.3031416

2020-01-01

Abstract:Visual content analysis and understanding attract tremendous attention because of its potentially wide range of applications including human activity analysis, automated photo face tagging, multicamera tracking, crowded counting, and biometric security. With recent progress in end-to-end differentiable learning, the accuracy of algorithms has been significantly improved and even outperforms humans in some tasks. In addition, multimodality methods, targeting on making full use of various visual data sources, are further investigated. These developments contribute to the innovations of two core modules for a typical intelligent vision system, i.e., image and video description and recognition, which are critical for the success of the visual content analysis and understanding in more complex and challenging open world.
Guest Editorial: Visual Analytics in Multimedia—Opportunities and Research Challenges

Nan Cao,Yingcai Wu,David Gotz,D. Kiem,Yap-Peng Tan

DOI: https://doi.org/10.1109/tmm.2016.2615400

IF: 7.3

2016-01-01

IEEE Transactions on Multimedia

Abstract:The ten papers in this special section are devoted to the topic of visual analytics, an emerging research direction that focuses on data exploration and analysis with a seamless integration of interaction, visualization, and analysis.
Guest Editorial: AI for Computational Audition—sound and Music Processing

Zijin Li,Wenwu Wang,Kejun Zhang,Mengyao Zhu

DOI: https://doi.org/10.1186/s13636-024-00353-7

2024-01-01

Abstract:Nowadays, the application of artificial intelligence (AI) algorithms and techniques is ubiquitous and transversal. Fields that take advantage of AI advances include sound and music processing. The advances in interdisciplinary research potentially yield new insights that may further advance the AI methods in this field. This special issue aims to report recent progress and spur new research lines in AI-driven sound and music processing, especially within interdisciplinary research scenarios.
Editorial IEEE Transactions on Multimedia Special Section on Video Analytics: Challenges, Algorithms, and Applications.

B. Prabhakaran,Y. -G. Jiang,H. Kalva,S. -F. Chang

DOI: https://doi.org/10.1109/tmm.2018.2821959

IF: 7.3

2018-01-01

IEEE Transactions on Multimedia

Abstract:The papers in this special section focus on the topic of video analytics. Also known as video content analysis, video analytics refer to the capability of automatically analyzing video to extract knowledge/information and detect and determine temporal and spatial events. The algorithms designed for these analytics can be implemented as software on general-purpose machines, or as hardware in specialized video processing units. Video analytics is still an emerging technology with techniques that are continuously being developed to help make widespread implementation feasible in the years ahead. Such analytics has been typically used in semantic categorization and retrieval of video databases. A goal of this special issue is to focus on video analytics beyond categorization and retrieval. With increasing hardware capability and advances in algorithms used, real-time video analytics is now being used in a wide range of domains including entertainment, health-care, retail, automotive, transport, home automation, emotion analysis, aesthetics, inappropriate content detection, safety and security. For instance, video analytics are increasingly being deployed for real-time alerts in situation monitoring systems such as traffic surveillance (vehicle counting), counting people in lines (some hospitals are using this to get more nurses from a less busy department to serve thewaiting patients) and in manufacturing (for monitoring and counting). From the sensing aspect, 3-D cameras such as RGB-D and LiDAR (Light Detection and Ranging) cameras are becoming more and more affordable, enabling additional areas of research and applications, such as self-driving cars employing video analytics on LiDAR captured data for path planning as well as obstacle detection.
Guest Editorial Advances in Generative Visual Signal Coding and Processing

Zhibo Chen,Heming Sun,Li Zhang,Fan Zhang

DOI: https://doi.org/10.1109/jetcas.2024.3403318

IF: 5.877

2024-01-01

IEEE Journal on Emerging and Selected Topics in Circuits and Systems

Abstract:This special issue of IEEE Journal on Emerging and Selected Topics in Circuits and Systems (JETCAS) is dedicated to demonstrating the latest developments in algorithms, implementations, and applications related to visual signal coding and processing with generative models. In recent years, generative models have emerged as one of the most significant and rapidly developing areas of research in artificial intelligence. They have proved to be an important instrument for advancing research in AI-based visual signal coding and processing. For instance, the variational autoencoder (VAE) has been used as a fundamental framework for end-to-end learned image coding, the autoregressive (AR) model has been extensively studied for efficient entropy coding, and the generative adversarial network (GAN) has been utilized frequently to enhance the subjective quality of coding schemes. Meanwhile, generative models have also been explored in various visual signal processing tasks, including quality assessment, restoration, enhancement, editing, and interpolation.
Editorial Applied Artificial Intelligence and Machine Learning for Video Coding and Streaming

Marta Mrak Marta,Mahmoud Reza Hashemi,Shervin Shirmohammadi,Ying Chen,Moncef Gabbouj,Marta Mrak,Mahmoud Hashemi

DOI: https://doi.org/10.1109/ojsp.2021.3105305

2021-01-01

IEEE Open Journal of Signal Processing

Abstract:The papers in this special section focus on applied artificial inteligence and machine learning for video coding and media streaming.
Guest Editorial Special Section on Visual Saliency Computing and Learning.

Junwei Han,Ling Shao,Nuno Vasconcelos,Jungong Han,Dong Xu

DOI: https://doi.org/10.1109/TNNLS.2016.2522738

IF: 14.255

2016-01-01

IEEE Transactions on Neural Networks and Learning Systems

Abstract:Vision and multimedia communities have long attempted to enable computers to understand image or video content in a manner analogous to humans. Humans’ comprehension to an image or a video clip often depends on the objects that draw their attention. As a result, one fundamental and open problem is to automatically infer the attention attracting or interesting areas in an image or a video sequence....
Editorial Advancements in Learning-Based Quality Prediction for Advanced Visual Media

Sebastian Bosse

DOI: https://doi.org/10.1109/jstsp.2023.3337626

IF: 7.695

2024-02-02

IEEE Journal of Selected Topics in Signal Processing

Abstract:In this special issue of the IEEE Journal of Selected Topics in Signal Processing, we delve into the burgeoning domain of Learning-Based Quality Prediction for Advanced Visual Media. The rapid proliferation of advanced visual media modalities, such as high dynamic range and mixed reality, has not only enhanced interactive and immersive user experiences but has also posed significant challenges in the realm of quality assessment and optimization. This issue collates pioneering research that addresses these challenges, underscoring the critical role of human perception in the acceptance and success of such applications.

engineering, electrical & electronic
Refilming with Depth-Inferred Videos

Guofeng Zhang,Zilong Dong,Jiaya Jia,Liang Wan,Tien-Tsin Wong,Hujun Bao

DOI: https://doi.org/10.1109/TVCG.2009.47

IF: 5.2

2009-01-01

IEEE Transactions on Visualization and Computer Graphics

Abstract:Compared to still image editing, content-based video editing faces the additional challenges of maintaining the spatiotemporal consistency with respect to geometry. This brings up difficulties of seamlessly modifying video content, for instance, inserting or removing an object. In this paper, we present a new video editing system for creating spatiotemporally consistent and visually appealing refi...
A Survey on High Coherence Visual Media Retargeting: Recent Advances and Applications

Weimin Tan,Bo Yan

DOI: https://doi.org/10.1007/s11704-016-6084-3

IF: 2.6688

2016-01-01

Frontiers of Computer Science

Abstract:The numerous works on media retargeting call for a thorough and comprehensive survey for reviewing and categorizing existing works and providing insights that can help future design of retargeting approaches and its applications. First, we present the basic problem of media retargeting and detail state-of-the-art retargeting methods devised to solve it. Second, we review recent works on objective quality assessment of media retargeting, where we find that although these works are designed to make the objective assessment result in accordance with the subjective evaluation, they are only suitable for certain situations. Considering the subjective nature of aesthetics, designing objective assessment metric for media retargeting could be a promising area for investigation. Third, we elaborate on other applications extended from retargeting techniques. We show how to apply the retargeting techniques in other fields to solve their challenging problems, and reveal that retargeting technique is not just a simple scaling algorithm, but a thought or concept, which has great flexibility and is quite useful.We believe this review can help researchers and practitioners to solve the existing problems of media retargeting and bring new ideas in their works.
Guest Editorial Introduction to the Special Section on Representation Learning for Visual Content Understanding

Jiwen Lu,Yuxin Peng,Guo-Jun Qi,Jun Yu

DOI: https://doi.org/10.1109/tcsvt.2020.3009095

IF: 5.859

2020-09-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Representation learning methods allow a system to automatically learn robust and discriminative features from raw data for given goals, which play an important role in various visual content understanding applications, such as visual object segmentation, detection, tracking, recognition, and search. The performance of visual content understanding tasks is heavily dependent on the choice of data representation (or features) on which they are applied. Conventional feature representation methods usually employ transformations of data that make it easier to extract useful information, such as scale-invariant feature transform (SIFT), local binary patterns (LBP), and histogram of oriented gradients (HOG). In recent years, deep learning techniques have been widely applied to learn data-driven representations with supervised annotations and achieved great success in different visual content understanding tasks. Representative methods include the ResNet method for image classification, the DeepFace method for face recognition, and the feature pyramid networks (FPNs) method for object detection. Despite recent progresses on deep representation learning with a great amount of annotated data, how to effectively learn visual representation with limited data annotations still requires many efforts. This special section focuses on data-effective representation learning methods for visual content understanding.

engineering, electrical & electronic
Intelligent Visual Media Processing: when Graphics Meets Vision.

Ming-Ming Cheng,Qi-Bin Hou,Song-Hai Zhang,Paul L. Rosin

DOI: https://doi.org/10.1007/s11390-017-1681-7

2017-01-01

Abstract:The computer graphics and computer vision communities have been working closely together in recent years, and a variety of algorithms and applications have been developed to analyze and manipulate the visual media around us. There are three major driving forces behind this phenomenon: 1) the availability of big data from the Internet has created a demand for dealing with the ever-increasing, vast amount of resources; 2) powerful processing tools, such as deep neural networks, provide effective ways for learning how to deal with heterogeneous visual data; 3) new data capture devices, such as the Kinect, the bridge between algorithms for 2D image understanding and 3D model analysis. These driving forces have emerged only recently, and we believe that the computer graphics and computer vision communities are still in the beginning of their honeymoon phase. In this work we survey recent research on how computer vision techniques benefit computer graphics techniques and vice versa, and cover research on analysis, manipulation, synthesis, and interaction. We also discuss existing problems and suggest possible further research directions.
A Survey on Content-Aware Video Analysis for Sports

Huang-Chia Shih

DOI: https://doi.org/10.48550/arXiv.1703.01170

2017-03-03

Computer Vision and Pattern Recognition

Abstract:Sports data analysis is becoming increasingly large-scale, diversified, and shared, but difficulty persists in rapidly accessing the most crucial information. Previous surveys have focused on the methodologies of sports video analysis from the spatiotemporal viewpoint instead of a content-based viewpoint, and few of these studies have considered semantics. This study develops a deeper interpretation of content-aware sports video analysis by examining the insight offered by research into the structure of content under different scenarios. On the basis of this insight, we provide an overview of the themes particularly relevant to the research on content-aware systems for broadcast sports. Specifically, we focus on the video content analysis techniques applied in sportscasts over the past decade from the perspectives of fundamentals and general review, a content hierarchical model, and trends and challenges. Content-aware analysis methods are discussed with respect to object-, event-, and context-oriented groups. In each group, the gap between sensation and content excitement must be bridged using proper strategies. In this regard, a content-aware approach is required to determine user demands. Finally, the paper summarizes the future trends and challenges for sports video analysis. We believe that our findings can advance the field of research on content-aware video analysis for broadcast sports.
Generative Visual Compression: A Review

Bolin Chen,Shanzhi Yin,Peilin Chen,Shiqi Wang,Yan Ye

2024-02-03

Abstract:Artificial Intelligence Generated Content (AIGC) is leading a new technical revolution for the acquisition of digital content and impelling the progress of visual compression towards competitive performance gains and diverse functionalities over traditional codecs. This paper provides a thorough review on the recent advances of generative visual compression, illustrating great potentials and promising applications in ultra-low bitrate communication, user-specified reconstruction/filtering, and intelligent machine analysis. In particular, we review the visual data compression methodologies with deep generative models, and summarize how compact representation and high-fidelity reconstruction could be actualized via generative techniques. In addition, we generalize related generative compression technologies for machine vision and intelligent analytics. Finally, we discuss the fundamental challenges on generative visual compression techniques and envision their future research directions.

Computer Vision and Pattern Recognition,Image and Video Processing
A Comprehensive Survey on 3D Content Generation

Jian Liu,Xiaoshui Huang,Tianyu Huang,Lu Chen,Yuenan Hou,Shixiang Tang,Ziwei Liu,Wanli Ouyang,Wangmeng Zuo,Junjun Jiang,Xianming Liu

2024-03-19

Abstract:Recent years have witnessed remarkable advances in artificial intelligence generated content(AIGC), with diverse input modalities, e.g., text, image, video, audio and 3D. The 3D is the most close visual modality to real-world 3D environment and carries enormous knowledge. The 3D content generation shows both academic and practical values while also presenting formidable technical challenges. This review aims to consolidate developments within the burgeoning domain of 3D content generation. Specifically, a new taxonomy is proposed that categorizes existing approaches into three types: 3D native generative methods, 2D prior-based 3D generative methods, and hybrid 3D generative methods. The survey covers approximately 60 papers spanning the major techniques. Besides, we discuss limitations of current 3D content generation techniques, and point out open challenges as well as promising directions for future work. Accompanied with this survey, we have established a project website where the resources on 3D content generation research are provided. The project page is available at

Computer Vision and Pattern Recognition,Artificial Intelligence
Guest Editorial Multimodal Modeling and Analysis Informed by Brain Imaging—Part II

Junwei Han,Tianming Liu,Christine Guo,Juyang Weng

DOI: https://doi.org/10.1109/tamd.2015.2498458

2015-01-01

IEEE Transactions on Autonomous Mental Development

Abstract:Human brains are the ultimate recipients and assessors of multimedia contents and semantics. Recent developments of neuroimaging techniques have enabled us to probe human brain activities during free viewing of multimedia contents. This special issue mainly focuses on the synergistic combinations of cognitive neuroscience, brain imaging, and multimedia analysis. It aims to capture the latest advances in the research community working on brain imaging informed multimedia analysis, as well as computational model of the brain processes driven by multimedia contents. This issue contains the second set of papers from the double Special Issue.
Introduction to the Special Issue on AI-Generated Content for Multimedia

Shengxi Li,Xuelong Li,Leonardo Chiariglione,Jiebo Luo,Wenwu Wang,Zhengyuan Yang,Danilo Mandic,Hamido Fujita

DOI: https://doi.org/10.1109/tcsvt.2024.3427488

IF: 5.859

2024-08-18

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:Our world is becoming rapidly dependent on data of increasing complexity, diversity, and volume which calls for robust and powerful tools to process such big data. Probabilistic generative models fulfill this goal by learning latent characteristic data relations, especially for the recent emergence of large-scale deep generative models that are able to create realistic content, namely, artificial intelligence-generated content (AIGC). The applications of AIGC span across various domains, and witness rich potential in multimedia content creation, including dialog generation, text-to-speech conversion, image/video generation, and cross-modal content generation.

engineering, electrical & electronic
Visual Communications and Image

Ya-Qin Zhang

2018-01-01

Abstract:Video and multimedia communication is a natural evolution from conventional voice communication. The last decade has witnessed phenomenal progress in the field in terms of research and development, products, services, and standardization activities. Advances in new image processing algorithms, video/multimedia communication standards, computer technologies VLSI devices, and fiber optics technologies will definitely continue to grow rapidly in association with global information superhighway construction and will create many valuable video services to the vast number of customers. Therefore, this fifth special section on visual communications and image processing is published to update the progress in this field. A total of 18 papers has been collected for this special section. Video/image coding still shows a strong interest with eight papers in this category covering standard activities; automata; vector quantization; and subband, wavelet, and object-oriented coding. Five papers deal with motion estimation issues. The remaining papers deal with image processing algorithms and applications. The two invited papers provide excellent overviews of the two important standards activities in audio-visual coded representation undertaken by 1S0 and ITU. Reader, the chairman of the ISO/MPEG4, discusses and outlines the background, vision, applications, scope of work, and workplan of MPEG4, a continuing effort of the ISO/WG 11 after its successful completion of MPEG1 and MPEG2. The paper by Schaphorst, the rapporteur for very low bitrate visual telephony for ITUTISG 15, describes the recent effort in H.324 to produce a video telephony standard in the Public Switched Telephone Network (PSTN) and the mobile environment. Quenneville and Meunier draw a parallel between quadtree and rational languages that are recognizable by finite state automata in their paper. The authors illustrate that the automata construction by recursive decomposition of the image is similar to the construction of region-based quadtree. ExProcessing
Editorial for Special Issue on Artificial Intelligence for Art

Luntian Mou,Feng Gao,Zijin Li,Jiaying Liu,Hongxun Yao,Johan F. Hoorn

DOI: https://doi.org/10.1007/s11633-023-1386-z

2024-01-01

Abstract:As an innovative engine for digital content generation,AI-generated content(AIGC)has drawn more and more attention from academic fields as well as industries.Spe-cifically in the area of art creation,AI has demonstrated its great potential and gained increasing popularity.People are greatly impressed by AI painting,composing,writing,virtual hosting,fashion,and design.Moreover,AI is also becoming capable of understanding art,evalu-ating the aesthetic value of art,and protecting the copy-right of art as well.AI has not only exhibited creativity to some extent,but also served as an enabling tool to dis-cover the principles underneath creativity and imagina-tion,which are traditional challenges for neuroscience,cognitive science,and psychology.Despite all these prom-ising features of AI for art,we still have to face the many challenges such as the explainability of generative models and the copyright issues of AI art works.
Guest Editorial Special Issue on Adversarial Learning in Computational Intelligence

Zixing Zhang,Dimitris n. Metaxas,Hung-Yi Lee,Bjorn W. Schuller

DOI: https://doi.org/10.1109/tetci.2020.3006295

2020-01-01

Abstract:The seven papers in this special section focus on adversarial learning in computational intelligence. The papers aim to capture the most recent advances of adversarial learning from both theoretical and empirical perspectives. Moreover, it attempts to present its novel applications to other domains beyond image generation. Adversarial learning has attracted tremendous attention in the community of machine learning over the past few years. It normally integrates two components that contest with each other in a two-player zero-sum game. Since its birth in 2014, adversarial learning has been widely applied to not only the generation of realistic images, but also many other research topics, such as data augmentation, domain adaptation, and adversarial attack, often leading to appealing performance. However, we have just witnessed the early rise of this technique, and still confront many challenges, for example, the mode collapse problem, and the interpretability of its results and failures. Computational Intelligence (CI) technologies are expected to provide efficient solutions to deal with the raised challenges. Moreover, most of the previous adversarial-learning studies are largely limited in addressing static images or feature vectors. It still remains largely an open question of how adversarial learning performs for other complex and temporally variational signals or modalities, such as speech and text.

Guest Editorial: Content-Aware Visual Systems: Analysis, Streaming, and Retargeting

Guest Editorial Introduction to the Special Section on Intelligent Visual Content Analysis and Understanding

Guest Editorial: Visual Analytics in Multimedia—Opportunities and Research Challenges

Guest Editorial: AI for Computational Audition—sound and Music Processing

Editorial IEEE Transactions on Multimedia Special Section on Video Analytics: Challenges, Algorithms, and Applications.

Guest Editorial Advances in Generative Visual Signal Coding and Processing

Editorial Applied Artificial Intelligence and Machine Learning for Video Coding and Streaming

Guest Editorial Special Section on Visual Saliency Computing and Learning.

Editorial Advancements in Learning-Based Quality Prediction for Advanced Visual Media

Refilming with Depth-Inferred Videos

A Survey on High Coherence Visual Media Retargeting: Recent Advances and Applications

Guest Editorial Introduction to the Special Section on Representation Learning for Visual Content Understanding

Intelligent Visual Media Processing: when Graphics Meets Vision.

A Survey on Content-Aware Video Analysis for Sports

Generative Visual Compression: A Review

A Comprehensive Survey on 3D Content Generation

Guest Editorial Multimodal Modeling and Analysis Informed by Brain Imaging—Part II

Introduction to the Special Issue on AI-Generated Content for Multimedia

Visual Communications and Image

Editorial for Special Issue on Artificial Intelligence for Art

Guest Editorial Special Issue on Adversarial Learning in Computational Intelligence