Proceedings of the ACM International Conference on Multimedia
Kien A. Hua,Yong Rui,Ralf Steinmetz,Alan Hanjalic,Apostol (Paul) Natsev,Wenwu Zhu
DOI: https://doi.org/10.1145/2647868
2014-01-01
Abstract:It is our pleasure to introduce to you the Technical Program of the ACM International Conference on Multimedia 2014. While we built this year's program upon the best practices from previous editions of the conference, we also introduced several adjustments with the goal to define the set of submission areas to reflect the current and emerging research directions in the multimedia field and to improve the quality of the review process and the feedback provided to the authors. Papers were submitted to 14 areas: Big and Broad Multimedia, Deep Learning for Multimedia, Emotional and Social Signals in Multimedia, Media Transport and Delivery, Multimedia and Society, Multimedia and the Crowd, Multimedia Art, Entertainment and Culture, Multimedia HCI and QoE, Multimedia Search and Recommendation, Multimedia Security, Privacy and Forensics, Multimedia Systems and Middleware, Multimodal Analysis and Description, Music, Speech and Audio Processing in Multimedia, and Social Media and Collective Online Presence. This set of areas resulted from revising and expanding the areas from previous years, and based on the discussion within the Technical Program Committee (TPC) that aimed at highlighting new and emerging research directions (e.g., big and broad multimedia, multimedia and society, QoE), promising new technical approaches not traditionally covered at ACM Multimedia (e.g., deep learning), and reaching out to related communities with complementary expertise (e.g., emotional and social signals). At the same time, some areas were discontinued, either due to the sub-standard number of submissions over the past several years (e.g. multimedia authoring and collaboration) or due to the fact that some technologies and platforms have become mainstream and need no special emphasis any more (e.g. mobile and multi-device). For each area, a team of Area Chairs (ACs) was appointed to handle the paper review process. In total, 38 ACs were appointed. The sizes of the teams were defined based on the anticipated number of submissions per area. Consequently, the teams ranged from two to seven ACs. The TPC consisted of both experienced and junior ACs. Also, some ACs were new to the ACM Multimedia venue, with the intention to broaden the ACM Multimedia community and open it up to related communities. ACs were given autonomy to define the Call for Papers (CfP) for their areas, select reviewers, assign papers to them (either manually or through a bidding process), assess papers based on their reviews, and provide initial acceptance recommendation as input for the TPC meeting. The activities of the ACs were steered at the meta level by the following guidelines defined by the TPC Chairs to maximize the program quality: The conference must preserve its unique multimedia character in order to stand out among related scientific venues and attract highest-quality submissions. The papers were handled in a double-blind fashion and for each paper a minimum of three reviews were required. The CfPs for different areas were defined so as to minimize overlap among areas, optimally highlight the focus of individual areas, and maximize the overall topic coverage of the conference program. For this purpose, ACs were requested to share their CfPs with other ACs, and to collaboratively resolve any overlap or ambiguity issues. This resulted in several area-CfP adjustments. ACs were also requested to share their lists of reviewers so as to maximize the level of expertise per area and to make the lists non-overlapping in order to control the review load per reviewer (a reviewer was allowed to participate in multiple areas only with the explicit agreement of the reviewer and the corresponding ACs). ACs were instructed to administratively reject improperly formatted papers and to move papers to other tracks if they believed there was a better fit for the paper. This required the agreement of ACs from both the outgoing and the receiving area. Conflict of Interest (CoI) handling: ACs were allowed to submit at most one full paper and one short paper to their own area. Since the submission system (EasyChair) only allowed the ACs to have access to papers and reviews in their own track, any submission by ACs were moved to a separate meta track, which included only papers with conflicts of interest. All of these papers were handled by another qualified AC, either from the same area or from another area. If the papers co-authored by ACs passed the threshold for discussion during the TPC meeting, the discussion was led by the handling AC and the author-AC was asked to leave the room. An elaborate EasyChair Paper Management Guidelines document was written containing instructions for the ACs on how to implement the TPC Chair's guidelines and how to navigate the submission management system (EasyChair). This Guidelines document will be made available to the SIGMM Director of Conferences to share with future TPCs of the ACM Multimedia conference. As an aside, we would strongly suggest to future TPCs to explore alternative conference management systems as EasyChair presented a number of challenges over the course of this process. Regarding the paper submission and review process, There was no pre-submission of abstracts. This led to a simplification of the submission process, and given the large number of submissions, we feel it did not negatively influence the paper submission process. Submission of full and short papers were separated from each other, with short papers being submitted only after the full paper submission had been closed. This improved the ease of managing different types of submissions, and allowed authors to more easily plan for both full and short papers, if desired. This may be a possible explanation for the noticeable increase in the number of submissions this year. For full papers, we organized a two-stage review process, and authors were invited to submit a rebuttal based on the first stage reviews. The reviewers were then instructed to consider the rebuttal, to revise their reviews accordingly, and to enter discussion with each other aiming to improve review consensus on each paper. The results of the second round reviews and discussion were then summarized by the ACs into meta reviews, which included tentative ACCEPT/REJECT/BORDERLINE recommendations, to be discussed and finalized during the TPC meeting. Compared to 2013, the full paper submission deadline was moved back to the end of March, while keeping the two-stage review process. Despite the significantly shortened review period, all full papers received at least three reviews on time (some as many as six!). We really appreciate the effort by our ACs and reviewers and thank them for the great job they did! Following the successful experience from 2013, an Author's Advocate (AA) role was created to provide authors with an independent channel to express concerns about the quality of the reviews for their papers, and to engage into a discussion with ACs on how to handle these concerns. The AA received requests for mediation for a total of 22 of the full papers (8%). The AA reviewed all the cases and successfully handled them in a collaborative effort with the ACs in charge of the corresponding papers. As a result of the mediation, 4 new reviews were generated and 14 other reviews were revised. The mediation from the AA happened before the TPC meeting, so the material serving as input for the discussions during the meeting did not include any low-quality review. We received 300 full and 397 short papers by the corresponding submission deadlines, or in total 697 papers. After removing improperly anonymized or otherwise invalid papers, 286 full and 393 short papers remained and entered the regular review process. The latter numbers served as the reference for computing the acceptance rate. Based on the first review results for full papers (after notification), 14 papers were withdrawn by the authors and removed from the system, leaving 665 papers in the review processing pipeline: 272 full papers and 393 short papersThis is the number that served as input for TPC meeting. The table below gives a distribution of these papers across areas. Decisions on full and short papers were made at the two-day TPC meeting held at Google Inc. in Mountain View, CA, USA on 18-19 June 2014. Two TPC Chairs and 28 ACs attended the meeting. While not all ACs were requested to be present, it was required that all areas are represented in the meeting. AC-teams per area were requested to agree on their recommendations on all papers in the area before the meeting so the representing AC(s) can lead the discussion for the entire area. On the first day, full papers were handled, and on the second day the short papers together with the discussion on awards nominees. We accepted 55 full and 117 short papers, leading to acceptance rates of 19.2% and 29.8% for full and short papers, respectively. The discussion on both full and short papers was conducted per area, leaving the ACs the opportunity to nominate the papers from their area into the pool of accepted papers. The discussion was at all times plenary, enabling ACs from any area to comment on any paper being discussed from the point of view of their individual expertise, experience, or rating criteria. The discussion went over all areas, and covered all papers flagged for discussion from each area by the ACs, as well as all papers scoring above a certain threshold in terms of their average review score. This way, we made sure to discuss both high-scoring and borderline papers as well as any controversial papers (with large review score discrepancy or with discrepancy between the scores and recommendations from the ACs). The discussion on the papers was intensive, taking into account all the information collected per paper, including reviews, the scores, the authors' rebuttals, the initial recommendations by the ACs (in the form of a meta review) and the reviews performed by other ACs during the meeting. After the meeting the TPC Chairs finalized the program by grouping the accepted papers into sessions, and the ACs finalized the meta reviews to include extra information based on the TPC meeting discussion. The sessions were formed based on research topics and do not necessarily reflect the submission areas. After notification, all accepted full papers were shepherded to make sure that the revised papers adequately addressed reviewers' concerns and the authors implemented any changes promised in their rebuttals. The TPC has done everything in their power to ensure that the technical program of the conference is of the highest possible quality. This would not have been possible without the commitment of the Area Chairs and the reviewers. We would like to thank them for their rigor and engagement, and we look forward to having a strong and engaging technical program in Orlando, FL! We hope you enjoy the program!