Abstract:Distant viewing approaches have typically used image datasets close to the contemporary image data used to train machine learning models. To work with images from other historical periods requires expert annotated data, and the quality of labels is crucial for the quality of results. Especially when working with cultural heritage collections that contain myriad uncertainties, annotating data, or re-annotating, legacy data is an arduous task. In this paper, we describe working with two pre-annotated sets of medieval manuscript images that exhibit conflicting and overlapping metadata. Since a manual reconciliation of the two legacy ontologies would be very expensive, we aim (1) to create a more uniform set of descriptive labels to serve as a "bridge" in the combined dataset, and (2) to establish a high quality hierarchical classification that can be used as a valuable input for subsequent supervised machine learning. To achieve these goals, we developed visualization and interaction mechanisms, enabling medievalists to combine, regularize and extend the vocabulary used to describe these, and other cognate, image datasets. The visual interfaces provide experts an overview of relationships in the data going beyond the sum total of the metadata. Word and image embeddings as well as co-occurrences of labels across the datasets, enable batch re-annotation of images, recommendation of label candidates and support composing a hierarchical classification of labels.

Ontology-Based Annotation of Multimedia Language Data for the Semantic Web

Building a Large Annotation Ontology for Movie Video Retrieval.

Multimedia ontology modeling: An approach based on MPEG-7

Improving Web-Based Learning: Automatic Annotation of Multimedia Semantics and Cross-Media Indexing

Semantic Annotation and Search for Educational Resources Supporting Distance Learning

Video Semantic Content Analysis Based on Ontology

Ontology-Based Annotation for Deep Web Data

Towards Structured Semantic Embedding of Multimedia

An Ontology-Based And Cooperative Annotation System

A Semantic Medical Multimedia Retrieval Approach Using Ontology Information Hiding

Probe into OWL Based Multimedia Ontology Building

Is Medieval Distant Viewing Possible? : Extending and Enriching Annotation of Legacy Image Collections using Visual Analytics

The OLAC Metadata Set and Controlled Vocabularies

A Survey of Semantic Image Annotation Tools Towards Digital Humanities

ImageSpace: An Environment for Image Ontology Management

Multimedia ontology matching by using visual and textual modalities

Emergent semantics from folksonomies: a quantitative study

Ontology-based digital photo annotation using multi-source information

Semi-automatic Video Content Annotation

Enriching Music Mood Annotation By Semantic Association Reasoning

The Music Meta Ontology: a flexible semantic model for the interoperability of music metadata