Abstract:This paper describes methods for automatically associating faces detected from multimedia documents with their names presented in the surrounding metadata. We consider the task in the image matching (IM) framework, where external Web facial images are automatically retrieved as the gallery face set of the names in advance, and a detected face is assigned to one of the names, or none of them, according to the association score between the two kinds of faces and constraints. Several important issues are investigated within the IM framework. In collecting Web facial images, beyond the basic scheme that use a celebrity name purely as the query to crawl facial images, a context-assisted image search method is proposed to enhance the relevance and discriminability of the retrieved faces. In constraint formulation, we propose an assigning-thresholding (AT) pipeline to uniformly ensure that the name-face correspondence is strictly one-to-one, and set low confidence associations as null assignments. In association score computation, we propose methods that jointly consider IM with the well-established graph-based association (GA) method at different stages, aiming at producing more accurate scores to benefit the association. Based on these efforts, an Accu-IM method performing the association as accurate as possible and a Fast-IM method performing the association in real-time are respective proposed. Extensive experiments on datasets of captioned News images and Web videos both demonstrate the advantages of the proposed efforts individually and jointly, which consistently provide improvement gains under different settings when compared with state-of-the-art methods.

Automatic Naming of Speakers in Video via Name-Face Mapping.

Automatic Face Annotation in TV Series by Video/script Alignment

Learning to Name Faces

Cast2Face: Assigning Character Names Onto Faces in Movie with Actor-Character Correspondence

Audio-driven Talking Face Video Generation with Natural Head Pose

Context-Oriented Name-Face Association in Web Videos.

Robust Speaking Face Identification For Video Analysis

AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Persons

Automatic Face Naming by Learning Discriminative Affinity Matrices from Weakly Labeled Images.

AnyoneNet: Synchronized Speech and Talking Head Generation for Arbitrary Person

Video-driven state-aware facial animation

Automated Video Labelling: Identifying Faces by Corroborative Evidence

Robust Face-Name Graph Matching for Movie Character Identification.

Identity-Preserving Talking Face Generation with Landmark and Appearance Priors

Improving Automatic Name-Face Association Using Celebrity Images on the Web

Cast2Face

Synthesizing Talking Faces from Text and Audio: an Autoencoder and Sequence-to-Sequence Convolutional Neural Network

Labeling Faces with Names Based on the Name Semantic Network

Name-face Association with Web Facial Image Supervision

Automatic Facial Image Annotation and Retrieval by Integrating Voice Label and Visual Appearance

Automatic Name-Face Alignment to Enable Cross-Media News Retrieval