Abstract:The comic domain is rapidly advancing with the development of single-page analysis and synthesis models. However, evaluation metrics and datasets lag behind, often limited to small-scale or single-style test sets. We introduce a novel benchmark, CoMix, designed to evaluate the multi-task capabilities of models in comic analysis. Unlike existing benchmarks that focus on isolated tasks such as object detection or text recognition, CoMix addresses a broader range of tasks including object detection, speaker identification, character re-identification, reading order, and multi-modal reasoning tasks like character naming and dialogue generation. Our benchmark comprises three existing datasets with expanded annotations to support multi-task evaluation. To mitigate the over-representation of manga-style data, we have incorporated a new dataset of carefully selected American comic-style books, thereby enriching the diversity of comic styles. CoMix is designed to assess pre-trained models in zero-shot and limited fine-tuning settings, probing their transfer capabilities across different comic styles and tasks. The validation split of the benchmark is publicly available for research purposes, and an evaluation server for the held-out test split is also provided. Comparative results between human performance and state-of-the-art models reveal a significant performance gap, highlighting substantial opportunities for advancements in comic understanding. The dataset, baseline models, and code are accessible at <a class="link-external link-https" href="https://github.com/emanuelevivoli/CoMix-dataset" rel="external noopener nofollow">this https URL</a>. This initiative sets a new standard for comprehensive comic analysis, providing the community with a common benchmark for evaluation on a large and varied set.

Unconstrained Text Detection in Manga: a New Dataset and Baseline

Comics Datasets Framework: Mix of Comics datasets for detection benchmarking

MangaUB: A Manga Understanding Benchmark for Large Multimodal Models

Comic Text Detection and Recognition Based on Deep Learning

A Comprehensive Gold Standard and Benchmark for Comics Text Detection and Recognition

Text-aware balloon extraction from manga

Building a Manga Dataset "Manga109" with Annotations for Multimedia Applications

M2C: Towards Automatic Multimodal Manga Complement

CoMix: A Comprehensive Benchmark for Multi-Task Comic Understanding

Towards Fully Automated Manga Translation

Facial Landmark Detection for Manga Images

CNN-based segmentation of speech balloons and narrative text boxes from comic book page images

The Manga Whisperer: Automatically Generating Transcriptions for Comics

Panel-Page-Aware Comic Genre Understanding.

Progressive Deep Feature Learning for Manga Character Recognition Via Unlabeled Training Data

Automating Manga Character Analysis: A Robust Deep Vision-Transformer Approach to Facial Landmark Detection

Multimodal Transformer for Comics Text-Cloze

Comic MTL: optimized multi-task learning for comic book image analysis

MangaGAN: Unpaired Photo-to-Manga Translation Based on the Methodology of Manga Drawing

Artistic-style text detector and a new Movie-Poster dataset

Context-Informed Machine Translation of Manga using Multimodal Large Language Models