Miniaturized Five Fundamental Issues about Visual Knowledge
Yun-he PAN
DOI: https://doi.org/10.1631/fitee.2040000
IF: 2.526
2021-01-01
Frontiers of Information Technology & Electronic Engineering
Abstract:Cognitive psychology has for a long time considered that an important part of human memory is visual knowledge, which is used for conducting concrete thinking. Therefore, visual-based artificial intelligence (AI) is a subject that AI cannot bypass and is of great significance. As a continuation of the previous article “On visual knowledge” (Pan, 2019), in this paper we discuss five fundamental issues about visual knowledge. Cognitive psychology experiments have been carried out to distinguish the characteristics of visual knowledge from those of language knowledge: (1) Visual knowledge can express the dimensions, color, texture, spatial shape, and spatial relationships of an object; (2) Visual knowledge can convey the movement, speed, and temporal relationships of an object; (3) Visual knowledge can facilitate space-time transformation, manipulation, and reasoning of objects (for example, the transformation of shape, action, velocity, and scene, and various spatio-temporal analogies, associations, and predictions based on spatio-temporal reasoning results). The expression of structural features realized by computer graphics (CG) is well suited to express the manipulation and transformation of characteristics (1) and (3), but it is challenging to apply CG to simulated imaginative transformation because CG expresses in a geometric form rather than a visual concept. A visual concept should be composed of a prototype and a domain. For instance, apples have everchanging forms, but they must have one or several fundamental shapes and colors comprising what is known as a prototype. Within the prototype, a variety of apples constitute a range of transformation which has a boundary. Forms within the boundary belong to the category “apple,” beyond which they will become other fruits. This range of transformation is the scope of the concept of “apple.” Visual concepts have a hierarchical structure, which is a spatial organization containing sub-concepts. Moreover, visual concepts have an action structure, which should contain the typical motion and action categories of each subconcept in the structure. The visual proposition is the expression of the spatial and temporal relationships of a visual concept. The spatial relationship can be regarded as the scene structure, which describes locational relationships (e.g., up and down, left and right, front and rear), distance relationships, internal and external relationships, and size relationships in geometric patterns. The temporal relationship can be regarded as the dynamic structure that conveys timing patterns such as growth, displacement, movement, transformation, competition, synergy, and evolution. Visual narration consists of a group of visual propositions. For example, Frontiers of Information Technology & Electronic Engineering www.jzus.zju.edu.cn; engineering.cae.cn; www.springerlink.com ISSN 2095-9184 (print); ISSN 2095-9230 (online) E-mail: jzus@zju.edu.cn