Hierarchical Knowledge Graph Construction from Images for Scalable E-Commerce

Zhantao Yang,Han Zhang,Fangyi Chen,Anudeepsekhar Bolimera,Marios Savvides

2024-10-29

Abstract:Knowledge Graph (KG) is playing an increasingly important role in various AI systems. For e-commerce, an efficient and low-cost automated knowledge graph construction method is the foundation of enabling various successful downstream applications. In this paper, we propose a novel method for constructing structured product knowledge graphs from raw product images. The method cooperatively leverages recent advances in the vision-language model (VLM) and large language model (LLM), fully automating the process and allowing timely graph updates. We also present a human-annotated e-commerce product dataset for benchmarking product property extraction in knowledge graph construction. Our method outperforms our baseline in all metrics and evaluated properties, demonstrating its effectiveness and bright usage potential.

Artificial Intelligence

What problem does this paper attempt to address?

The paper attempts to address the problem of how to automatically generate structured knowledge graphs (Knowledge Graphs, KG) from product images in the field of e-commerce. Specifically, the paper proposes a novel method that utilizes the latest Vision-Language Models (VLMs) and Large Language Models (LLMs) to directly construct complex product knowledge graphs from raw product images. This method aims to achieve a fully automated process without human intervention, thereby enabling timely updates of the knowledge graph to adapt to the rapidly changing e-commerce environment.

The main contributions of the paper include:
1. **Proposing a novel method**: For the first time, a fully automated method is proposed to generate knowledge graphs using only product images.
2. **Introducing a benchmark dataset**: A dataset containing 105 annotated e-commerce product images is provided for evaluating the knowledge graph generation task.
3. **Superior performance**: The proposed method significantly outperforms baseline methods on multiple metrics, demonstrating its effectiveness and potential application value.

Through these contributions, the paper addresses several key challenges in knowledge graph construction in e-commerce, such as information extraction, attribute inference, and hierarchical expansion, thereby providing a more efficient and accurate knowledge representation foundation for e-commerce applications.