A Systematic Study on Object Recognition Using Millimeter-wave Radar

Maloy Kumar Devnath,Avijoy Chakma,Mohammad Saeid Anwar,Emon Dey,Zahid Hasan,Marc Conn,Biplab Pal,Nirmalya Roy

2023-05-03

Abstract:Due to its light and weather-independent sensing, millimeter-wave (MMW) radar is essential in smart environments. Intelligent vehicle systems and industry-grade MMW radars have integrated such capabilities. Industry-grade MMW radars are expensive and hard to get for community-purpose smart environment applications. However, commercially available MMW radars have hidden underpinning challenges that need to be investigated for tasks like recognizing objects and activities, real-time person tracking, object localization, etc. Image and video data are straightforward to gather, understand, and annotate for such jobs. Image and video data are light and weather-dependent, susceptible to the occlusion effect, and present privacy problems. To eliminate dependence and ensure privacy, commercial MMW radars should be tested. MMW radar's practicality and performance in varied operating settings must be addressed before promoting it. To address the problems, we collected a dataset using Texas Instruments' Automotive mmWave Radar (AWR2944) and reported the best experimental settings for object recognition performance using different deep learning algorithms. Our extensive data gathering technique allows us to systematically explore and identify object identification task problems under cross-ambience conditions. We investigated several solutions and published detailed experimental data.

Computer Vision and Pattern Recognition,Artificial Intelligence,Computational Engineering, Finance, and Science,Robotics

What problem does this paper attempt to address?

The paper aims to address the problem of object recognition using millimeter-wave radar (MMW radar) under different environmental conditions. Specifically, the researchers focus on how to effectively recognize small objects in various indoor and outdoor environments using commercially available, cost-effective millimeter-wave radar. Due to the limitations of traditional image and video data under lighting and weather conditions, as well as privacy issues, the researchers explored millimeter-wave radar as an alternative sensing mechanism.

The challenges faced in the study include:
1. **Object Size and Shape**: Small objects such as metal fragments, household pets, electronic devices, etc., have a small surface area, limiting their ability to receive and reflect radar signals.
2. **Operating Environment**: The reflective properties of different materials lead to variations in the signal-to-noise ratio, and signals reflected by surrounding objects can interfere with the final received signal, affecting the object recognition task.
3. **Radar Position and Distance**: The position of the radar and its distance from the target object significantly impact the recognition effect. As the distance increases, signal diffusion leads to a weakening of the received signal strength.

To overcome these challenges, the paper proposes the following contributions:
1. Developing models using low-resolution and sparse point cloud data to recognize small objects in three different indoor and outdoor environments.
2. Using domain adaptation techniques to enhance the robustness of deep learning models, especially when dealing with low-resolution and sparse point cloud data from millimeter-wave radar. To the best of the authors' knowledge, this is the first time domain adaptation has been applied to such object recognition research.
3. Validating the proposed system with a large-scale dataset in environments with ample sunlight, good indoor lighting, and dark indoor conditions, achieving an F1 score of 0.898.

In summary, the main objective of this paper is to evaluate the performance of commercial millimeter-wave radar in object recognition under different environmental conditions and to propose a domain adaptation-based solution to improve the accuracy of recognizing small objects.