Visualization of isomorphism-synesthesia of colour and music
Xue Mao,Jie Xu,Jiahong Lang,Shangshu Zhang
DOI: https://doi.org/10.1016/j.visinf.2023.10.006
2023-11-05
Visual Informatics
Abstract:Music and colour, as human hearing and visual art, are closely related to human psychological feelings and symbolic associations. There is an isomorphic relationship between music and colour. The article uses the concept of "synesthesia" in psychology and the "co-construction" relationship in mathematics as a bridge, based on Kandinsky's "inner sound" theory and Mallion's "tone-colour system", an interdisciplinary theoretical model of "timbre isomorphism synesthesia" (ISCM) is constructed. At the practical level, based on the ISCM theory, a set of timbre synesthesia visualization tool ASAH and visualization process are designed, through music data input, graphics mapping visualization, colour mapping visualization, real-time interactive visualization, and finally output timbre synesthesia visualization works. In order to avoid the visual homogenization caused by algorithm design, ASHA has set up a custom editor, which emphasizes the individual differences and multi-sensory experience of tonal synesthesia visualization.
computer science, information systems, interdisciplinary applications, software engineering