Abstract:Objects, in the real world, rarely occur in isolation and exhibit typical arrangements governed by their independent utility, and their expected interaction with humans and other objects in the context. For example, a chair is expected near a table, and a computer is expected on top. Humans use this spatial context and relative placement as an important cue for visual recognition in case of ambiguities. Similar to human's, DNN's exploit contextual information from data to learn representations. Our research focuses on harnessing the contextual aspects of visual data to optimize data annotation and enhance the training of deep networks. Our contributions can be summarized as follows: (1) We introduce the notion of contextual diversity for active learning CDAL and show its applicability in three different visual tasks semantic segmentation, object detection and image classification, (2) We propose a data repair algorithm to curate contextually fair data to reduce model bias, enabling the model to detect objects out of their obvious context, (3) We propose Class-based annotation, where contextually relevant classes are selected that are complementary for model training under domain shift. Understanding the importance of well-curated data, we also emphasize the necessity of involving humans in the loop to achieve accurate annotations and to develop novel interaction strategies that allow humans to serve as fact-checkers. In line with this we are working on developing image retrieval system for wildlife camera trap images and reliable warning system for poor quality rural roads. For large-scale annotation, we are employing a strategic combination of human expertise and zero-shot models, while also integrating human input at various stages for continuous feedback.

Data Curation with Deep Learning [Vision]

Deep Learning for Content-Based Image Retrieval: A Comprehensive Study

Data collection and quality challenges in deep learning: a data-centric AI perspective

A Vision for Semantically Enriched Data Science

Deep Learning for LiDAR-only and LiDAR-fusion 3D Perception: a Survey

A Survey of Deep Learning for Scientific Discovery

A Survey on Deep Learning for Big Data

A survey on Deep Learning Advances on Different 3D Data Representations

Deep Learning-Driven Data Curation and Model Interpretation for Smart Manufacturing

A Survey on Deep Learning: Algorithms, Techniques, and Applications.

Data-centric AI: Perspectives and Challenges

VisionKG: Unleashing the Power of Visual Datasets via Knowledge Graph

Deep Learning for Spatiotemporal Big Data: A Vision on Opportunities and Challenges

Cit: Curation in training for effective vision-language data

Deep-learning-based visual data analytics for smart construction management

Deep learning in computer vision: A critical review of emerging techniques and application scenarios

Deep Learning based Visually Rich Document Content Understanding: A Survey

Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers

Data Curation: Towards a Tool for All

Deep learning for visual understanding: A review

Exploiting Contextual Uncertainty of Visual Data for Efficient Training of Deep Models