Memory-Efficient High-Accuracy Food Intake Activity Recognition with 3D Mmwave Radars

Hsin-Che Chiang,Yi-Hung Wu,Shervin Shirmohammadi,Cheng-Hsin Hsu

DOI: https://doi.org/10.1145/3607828.3617791

2023-01-01

Abstract:Non-invasive and privacy-preserving recognition of food intake activities has applications in diet management, telecare, and smarthome data monetization. In this paper, we tackle the challenging problem of recognizing food intake activity using sparse point clouds captured by a single mmWave radar, which is non-invasive and privacy-preserving. We propose: (i) an enhanced Skeletal Pose Estimator (SPE) capable of generating more precise skeletons for food intake activity recognition, outperforming the state-of-the-art MARS with an average reduction of 45.16% in the error of the estimated joints, (ii) a Dynamic Point Cloud Recognizer (DPR), which adapts SPE to directly process dynamic point clouds for food intake activity recognition, outperforming the state-of-the-art FIA with a 4.10% enhancement in classification accuracy and a 78.29% reduction in memory consumption, and (iii) a Lightweight Dynamic Point Cloud Recognizer (LDPR), which eliminates the need for CNNs, hence reducing model complexity and outperforming DPR by 0.15% in classification accuracy and a 42.80% reduction in memory consumption. In addition to food intake activity recognition, the skeletons generated by our SPE can also be used to recognize other fine- and coarse-grained activities for applications like rehabilitation, driver monitoring, and fitness assistance.