A Multimodal Dangerous State Recognition and Early Warning System for Elderly with Intermittent Dementia

Liyun Deng,Lei Jin,Guangcheng Wang,Quan Shi,Han Wang

2024-05-30

Abstract:In response to the social issue of the increasing number of elderly vulnerable groups going missing due to the aggravating aging population in China, our team has developed a wearable anti-loss device and intelligent early warning system for elderly individuals with intermittent dementia using artificial intelligence and IoT technology. This system comprises an anti-loss smart helmet, a cloud computing module, and an intelligent early warning application on the caregiver's mobile device. The smart helmet integrates a miniature camera module, a GPS module, and a 5G communication module to collect first-person images and location information of the elderly. Data is transmitted remotely via 5G, FTP, and TCP protocols. In the cloud computing module, our team has proposed for the first time a multimodal dangerous state recognition network based on scene and location information to accurately assess the risk of elderly individuals going missing. Finally, the application software interface designed for the caregiver's mobile device implements multi-level early warnings. The system developed by our team requires no operation or response from the elderly, achieving fully automatic environmental perception, risk assessment, and proactive alarming. This overcomes the limitations of traditional monitoring devices, which require active operation and response, thus avoiding the issue of the digital divide for the elderly. It effectively prevents accidental loss and potential dangers for elderly individuals with dementia.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

### Problems the Paper Attempts to Solve

This paper aims to address the social issue of elderly dementia patients going missing due to the worsening aging problem in China. Specifically, the research team has developed a wearable anti-loss device and an intelligent warning system based on artificial intelligence and Internet of Things (IoT) technology for elderly people with intermittent dementia.

#### Main Issues
1. **The problem of elderly dementia patients going missing**:
- With the worsening aging problem in China, more and more elderly people are prone to getting lost due to memory decline.
- Existing monitoring devices usually require the elderly to operate or respond actively, which is difficult for those with impaired memory.

2. **Limitations of existing monitoring methods**:
- Current methods mainly rely on single indicators such as electronic fence settings or fall detection to assess the safety status of the elderly.
- These methods ignore the impact of multiple factors such as environment, time, and location on risk, lacking comprehensive risk assessment means.

#### Solution
The research team designed a multimodal danger state recognition method and warning system, including the following parts:

1. **Smart anti-loss helmet**:
- The helmet is equipped with a micro camera module, GPS module, and 5G communication module, which can collect first-person perspective image information and location information in real-time.
- Data is remotely transmitted to the server for analysis and processing through 5G, FTP, and TCP protocols.

2. **Multimodal danger state recognition network**:
- A cloud-based multimodal danger state recognition network based on scene and location information is constructed to comprehensively assess the danger state of the elderly.
- This network includes four key modules: Pseudo Map Generation Module (MGM), Unimodal Feature Extraction Module (UFE), Multimodal Feature Fusion Module (MFF), and Hazard State Recognition Module (HSR).

3. **Intelligent warning software**:
- An intelligent warning software based on the Android system is designed, which can display the first-person perspective image and location information of the elderly in real-time on mobile devices and trigger different levels of warnings based on the danger state recognition results.
- It can achieve automatic environment perception, risk assessment, and warning functions without the need for the elderly to operate.

Through the integrated application of the above system, it can effectively prevent accidental loss and potential dangers of elderly dementia patients.