Cultural heritage digital twin: modeling and representing the visual narrative in Leonardo Da Vinci's Mona Lisa

Alessia Amelio,Gian Piero Zarri
DOI: https://doi.org/10.1007/s00521-024-10010-x
2024-06-08
Neural Computing and Applications
Abstract:In this paper, Artificial Intelligence/Knowledge Representation methods are used for the digital modeling of cultural heritage elements. Accordingly, the new concept of digital cultural heritage twin is presented as composed of a physical component and an immaterial component of the cultural entity. The former concerns the physical aspects, i.e. style, name of the artist, execution time, dimension, etc. The latter represents the emotional and intangible aspects transmitted by the entity, i.e. emotions, thoughts, opinions. In order to digitally model the physical and immaterial components of the twin, the Narrative Knowledge Representation Language has been formally introduced and described. It is particularly suitable for representing the immaterial aspects of the cultural entity, as it is capable of modeling in a simple but rigorous and efficient way complex situations and events, behaviours, attitudes, etc. As an experiment, NKRL has been adopted for representing some of the most relevant intangible items of the visual narrative underlying the hidden painting that lies beneath the Mona Lisa (La Gioconda) image painted by Leonardo Da Vinci on the same poplar panel. Real-time application of the resulting knowledge base opens up novel possibilities for the development of virtual objects, chatbots and expert systems, as well as the definition of semantic search platforms related to cultural heritage.
computer science, artificial intelligence
What problem does this paper attempt to address?
The paper attempts to address the issue of insufficient modeling and representation of intangible and emotional information in the digitization process of cultural heritage. Specifically, existing cultural heritage digitization methods mainly focus on physical attributes (such as dimensions, materials, authors, etc.), while neglecting intangible and conceptual features (such as emotions, thoughts, cultural implications, etc.). To address this shortcoming, the paper proposes the concept of "Digital Cultural Heritage Twin," which includes both physical and intangible components. ### Main Contributions: 1. **Introduction of Digital Cultural Heritage Twin**: - It includes not only the physical quantifiable attributes of cultural heritage items but also intangible/conceptual features such as emotions, feelings, emotional associations, cultural inspirations, historical memories, etc. 2. **Adoption of Narrative Knowledge Representation Language (NKRL)**: - Uses n-ary knowledge representation methods instead of traditional binary methods to more effectively model and represent complex situations and events such as emotions and feelings. 3. **Extension of NKRL in the Cultural Heritage Field**: - Applies NKRL to the cultural heritage field, particularly exploring its capability in handling Iconographic Narratives, which are very important and challenging components in the cultural heritage domain. ### Background and Related Work: - **Early Cultural Heritage Digitization**: Since the 1960s, mainly described the physical attributes of cultural heritage through keywords/metadata. - **Development of Data Models**: In the 1980s and 1990s, data models specifically for cultural heritage needs emerged, such as Dublin Core and VRA, but these models were relatively simple in knowledge representation. - **CIDOC CRM**: As the main tool for cultural heritage documentation, it provides a formal ontology structure but only supports binary relationships, limiting its expressive capability. - **Impact of the Semantic Web**: After 2000, many models gradually migrated to RDF-compatible versions, improving interoperability and standardization potential, but still faced difficulties in expressing complex relationships. - **Digital Cultural Heritage Twin**: As a solution, the Digital Cultural Heritage Twin aims to comprehensively consider all semantic/conceptual/intangible aspects of cultural heritage, requiring the use of advanced n-ary foundational knowledge representation tools. ### Experiments and Results: - **Experimental Object**: Leonardo da Vinci's painting "Mona Lisa," with a particular focus on the hidden painting beneath it. - **Method**: Using NKRL to model the Digital Cultural Heritage Twin of the "Mona Lisa," focusing on the narrative of the hidden painting. - **Results**: Demonstrated the effectiveness and flexibility of NKRL in modeling complex emotions and narratives. ### Conclusion and Future Directions: - **Conclusion**: NKRL performs excellently in handling intangible and emotional information of cultural heritage, providing a more complete and accurate digital representation. - **Future Directions**: Further optimize the application of NKRL, extend it to more cultural heritage digitization projects, and develop applications such as virtual objects, chatbots, and expert systems.