Abstract:SafeAccess is an integrated system designed to provide easier and safer access to a smart home for people with or without disabilities. The system is designed to enhance safety and promote the independence of people with disability (i.e., visually impaired). The key functionality of the system includes the detection and identification of human and generating contextual visual summary from the real-time video streams obtained from the cameras placed in strategic locations around the house. In addition, the system classifies human into groups (i.e. friends/families/caregiver versus intruders/burglars/unknown). These features allow the user to grant/deny remote access to the premises or ability to call emergency services. In this paper, we focus on designing a prototype system for the smart home and building a robust recognition engine that meets the system criteria and addresses speed, accuracy, deployment and environmental challenges under a wide variety of practical and real-life situations. To interact with the system, we implemented a dialog enabled interface to create a personalized profile using face images or video of friend/families/caregiver. To improve computational efficiency, we apply change detection to filter out frames and use Faster-RCNN to detect the human presence and extract faces using Multitask Cascaded Convolutional Networks (MTCNN). Subsequently, we apply LBP/FaceNet to identify a person and groups by matching extracted faces with the profile. SafeAccess sends a visual summary to the users with an MMS containing a person's name if any match found or as "Unknown", scene image, facial description, and contextual information. SafeAccess identifies friends/families/caregiver versus intruders/unknown with an average F-score 0.97 and generates a visual summary from 10 classes with an average accuracy of 98.01%.

Preferences Prediction using a Gallery of Mobile Device based on Scene Recognition and Object Detection

A Convenient and Efficient Scene Shop Recognition Assistant Tool for the Visually Impaired

Efficient Gesture Recognition for the Assistance of Visually Impaired People using Multi-Head Neural Networks

Efficient Facial Representations for Age, Gender and Identity Recognition in Organizing Photo Albums using Multi-output CNN

Person Identification with Visual Summary for a Safe Access to a Smart Home

The 2013 Face Recognition Evaluation in Mobile Environment

Facial Emotion Recognition for Mobile Devices: A Practical Review

Some like it hot - visual guidance for preference prediction

Demographic Information Prediction: A Portrait of Smartphone Application Users

Android Based Object Detection System for Visually Impaired

Facial expression recognition based on adaptation of the classifier to videos of the user

Event-Based Gesture Recognition With Dynamic Background Suppression Using Smartphone Computational Capabilities

Multimodal Prediction and Personalization of Photo Edits with Deep Generative Models

MobilePortrait: Real-Time One-Shot Neural Head Avatars on Mobile Devices

Empowering Visually Impaired Individuals: A Novel Use of Apple Live Photos and Android Motion Photos

Photofeeler-D3: A Neural Network with Voter Modeling for Dating Photo Impression Prediction

Autoencoder-enabled eye-tracking data analytics for objective assessment of user preference in humanoid robot appearance design

XFace: A Face Recognition System for Android Mobile Phones

Development of Android Application for Gender, Age and Face Recognition Using OpenCV

DegustaBot: Zero-Shot Visual Preference Estimation for Personalized Multi-Object Rearrangement

Classifying Emotions and Engagement in Online Learning Based on a Single Facial Expression Recognition Neural Network