Abstract:Interactions within virtual environments often require manipulating 3D virtual objects. To this end, researchers have endeavoured to find efficient solutions using either traditional input devices or focusing on different input modalities, such as touch and mid‐air gestures. Different virtual environments and diverse input modalities present specific issues to control object position, orientation and scaling: traditional mouse input, for example, presents non‐trivial challenges because of the need to map between 2D input and 3D actions. While interactive surfaces enable more natural approaches, they still require smart mappings. Mid‐air gestures can be exploited to offer natural manipulations mimicking interactions with physical objects. However, these approaches often lack precision and control. All these issues and many others have been addressed in a large body of work. In this article, we survey the state‐of‐the‐art in 3D object manipulation, ranging from traditional desktop approaches to touch and mid‐air interfaces, to interact in diverse virtual environments. We propose a new taxonomy to better classify manipulation properties. Using our taxonomy, we discuss the techniques presented in the surveyed literature, highlighting trends, guidelines and open challenges, that can be useful both to future research and to developers of 3D user interfaces. Interactions within virtual environments often require manipulating 3D virtual objects. We survey the state‐of‐the‐art in 3D object manipulation, ranging from traditional desktop approaches to touch and mid‐air interfaces, to interact in diverse virtual environments. We propose a new taxonomy to better classify manipulation properties. Using our taxonomy, we discuss the techniques presented in the surveyed literature, highlighting trends, guidelines and open challenges, that can be useful both to future research and to developers of 3D user interfaces.

Mixed interaction: evaluating user interactions for object manipulations in virtual space

Influence of Hand Tracking as a Way of Interaction in Virtual Reality on User Experience

An interaction system using mixed hand gestures.

Evaluating Object Manipulation Interaction Techniques in Mixed Reality: Tangible User Interfaces and Gesture

Trade-Off between Task Accuracy, Task Completion Time and Naturalness for Direct Object Manipulation in Virtual Reality

Investigating the Effects of Avatarization and Interaction Techniques on Near-field Mixed Reality Interactions with Physical Components

User Experience of Virtual-Reality Interactive Interfaces: A Comparison Between Hand Gesture Recognition and Joystick Control for XRSPACE MANOVA

An evaluation of multimodal interaction techniques for 3D layout constraint solver in a desktop-based virtual environment

A Survey on 3D Virtual Object Manipulation: From the Desktop to Immersive Virtual Environments

Multimodal Interfaces for Effective Teleoperation

Comparing Different Grasping Visualizations for Object Manipulation in VR using Controllers

Real-time multimodal interaction in virtual reality - a case study with a large virtual interface

Two-In-One: A Design Space for Mapping Unimanual Input into Bimanual Interactions in VR for Users with Limited Movement

Exploring Data Input Problems in Mixed Reality Environments: Proposal and Evaluation of Natural Interaction Techniques

Eliciting Multimodal Gesture+Speech Interactions in a Multi-Object Augmented Reality Environment

Virtual reality interaction based on visual attention and kinesthetic information

User Experience Evaluation in Shared Interactive Virtual Reality

Evaluation of Point-to-Point Reaching Performance in Mixed Reality and Virtual Reality

Above Surface Interaction for Multiscale Navigation in Mobile Virtual Reality

VVIR-OM: Efficient Object Manipulation in VR with Variable Virtual Interaction Region

A comparative analysis of two immersive virtual reality systems in the integration and visualization of natural hand interaction