Abstract:With the uproar of touchless technology, the Virtual Continuum has seen some spark in the upcoming products. Today numerous gadgets support the use of Mixed Reality / Augmented Reality (AR)/ Virtual Reality. The Head Mounted Displays (HMDs) like that of Hololens, Google Lens, Jio Glass manifested reality into virtuality. Other than the HMDs many organizations tend to develop mobile AR applications to support umpteen number of industries like medicine, education, construction. Currently, the major issue lies in the performance parameters of these applications, while deploying for mobile application's graphics performance, latency, and CPU functioning. Many industries pose real-time computation requirements in AR but do not implement an efficient algorithm in their frameworks. Offloading the computation of deep learning models involved in the application to the cloud servers will highly affect the processing parameters. For our use case, we will be using Multi-Task Cascaded Convolutional Neural Network (MTCNN) which is a modern tool for face detection, using a 3-stage neural network detector. Therefore, the optimization of communication between local application and cloud computing frameworks needs to be optimized. The proposed framework defines how the parameters involving the complete deployment of a mobile AR application can be optimized in terms of retrieval of multimedia, its processing, and augmentation of graphics, eventually enhancing the performance. To implement the proposed algorithm a mobile application is created in Unity3D. The mobile application virtually augments a 3D model of a skeleton on a target face. After the mentioned experimentation, it is found that average Media Retrieval Time (1.1471 μ s) and Client Time (1.1207 μ s) in the local application are extremely low than the average API process time (288.934ms). The highest time latency is achieved at the frame rate higher than 80fps.

Optimizing the Learning Accuracy in Mobile Augmented Reality Systems with CNN

Optimizing the Learning Performance in Mobile Augmented Reality Systems with CNN

Deep Learning on Mobile and Embedded Devices: State-of-the-art, Challenges, and Future Directions

Modeling the Resource Requirements of Convolutional Neural Networks on Mobile Devices

Augur: Modeling the Resource Requirements of ConvNets on Mobile Devices

Deep learning model based multimedia retrieval and its optimization in augmented reality applications

An Edge-Computing Based Architecture for Mobile Augmented Reality

CroApp: A CNN-Based Resource Optimization Approach in Edge Computing Environment

Edge intelligence computing for mobile augmented reality with deep reinforcement learning approach

A Unified Optimization Approach for CNN Model Inference on Integrated GPUs

Accelerating Convolutional Neural Networks for Continuous Mobile Vision Via Cache Reuse.

Accurate Deep Learning Inference Latency Prediction over Dynamic Running Mobile Devices

Methodology to Deploy CNN-Based Computer Vision Models on Immersive Wearable Devices

Real-Time Facial Affective Computing on Mobile Devices

Smartphone-based Eye Tracking System using Edge Intelligence and Model Optimisation

Exploiting Sparsity to Accelerate Fully Connected Layers of CNN-Based Applications on Mobile SoCs

Optimization in Mobile Augmented Reality Systems for the Metaverse over Wireless Communications

Resource and Data Optimization for Hardware Implementation of Deep Neural Networks Targeting FPGA-based Edge Devices.

A Low-Power Hardware Architecture for Real-Time CNN Computing

DistrEdge: Speeding up Convolutional Neural Network Inference on Distributed Edge Devices

Energy Drain of the Object Detection Processing Pipeline for Mobile Devices: Analysis and Implications