From Virtual Reality to the Emerging Discipline of Perception Engineering

Steven M. LaValle,Evan G. Center,Timo Ojala,Matti Pouke,Nicoletta Prencipe,Basak Sakcak,Markku Suomalainen,Kalle G. Timperi,Vadim K. Weinstein

DOI: https://doi.org/10.1146/annurev-control-062323-102456

2024-03-27

Abstract:This paper makes the case that a powerful new discipline, which we term perception engineering, is steadily emerging. It follows from a progression of ideas that involve creating illusions, from historical paintings and film, to video games and virtual reality in modern times. Rather than creating physical artifacts such as bridges, airplanes, or computers, perception engineers create illusory perceptual experiences. The scope is defined over any agent that interacts with the physical world, including both biological organisms (humans, animals) and engineered systems (robots, autonomous systems). The key idea is that an agent, called a producer, alters the environment with the intent to alter the perceptual experience of another agent, called a receiver. Most importantly, the paper introduces a precise mathematical formulation of this process, based on the von Neumann-Morgenstern notion of information, to help scope and define the discipline. It is then applied to the cases of engineered and biological agents with discussion of its implications on existing fields such as virtual reality, robotics, and even social media. Finally, open challenges and opportunities for involvement are identified.

Human-Computer Interaction,Systems and Control

What problem does this paper attempt to address?

This paper discusses an emerging discipline called Perception Engineering. This field originated from the idea of creating illusions, from ancient paintings and films to modern virtual reality technology. The core of Perception Engineering is for designers (producers) to manipulate the environment to influence the perceptual experience of recipients (receivers), whether they are living organisms (such as humans or animals) or artificial systems (such as robots or autonomous systems).

The authors propose a precise mathematical representation based on the Von Neumann-Morgenstern information concept to define and delineate this field, and apply it to engineering and biological agents. They emphasize the importance of understanding how VR systems work, including displays, sensors, and biological principles such as perception psychology, neuroscience, and physiology.

In the paper, the mathematical model of Perception Engineering not only covers VR users wearing head-mounted displays, but also extends to a wider range of methods for creating perceptual experiences, such as machine learning, nanophotonics, and social media technology. The authors point out that although VR is currently mainly focused on technological implementation, there may be more methods developed in the future for creating perceptual illusions.

Furthermore, the paper discusses applications in robots and living organisms, as well as the potential extensions of Perception Engineering in social engineering, such as the spread of fake news on social media. The paper concludes by highlighting the challenges and future research opportunities in this field.

In summary, this paper aims to elucidate a new discipline, Perception Engineering, which influences the behavior of living organisms and machines by creating and manipulating perceptual experiences. This will have profound implications for fields such as virtual reality, robotics, and social interaction.