Abstract:Artificial Intelligence of Things (AIoT) is an emerging frontier based on the deep fusion of Internet of Things (IoT) and Artificial Intelligence (AI) technologies. Although advanced deep learning techniques enhance the efficient data processing and intelligent analysis of complex IoT data, they still suffer from notable challenges when deployed to practical AIoT applications, such as constrained resources, and diverse task requirements. Knowledge transfer is an effective method to enhance learning performance by avoiding the exorbitant costs associated with data recollection and model retraining. Notably, although there are already some valuable and impressive surveys on transfer learning, these surveys introduce approaches in a relatively isolated way and lack the recent advances of various knowledge transfer techniques for AIoT field. This survey endeavors to introduce a new concept of knowledge transfer, referred to as Crowd Knowledge Transfer (CrowdTransfer), which aims to transfer prior knowledge learned from a crowd of agents to reduce the training cost and as well as improve the performance of the model in real-world complicated scenarios. Particularly, we present four transfer modes from the perspective of crowd intelligence, including derivation, sharing, evolution and fusion modes. Building upon conventional transfer learning methods, we further delve into advanced crowd knowledge transfer models from three perspectives for various AIoT applications. Furthermore, we explore some applications of AIoT areas, such as human activity recognition, urban computing, multi-robot system, and smart factory. Finally, we discuss the open issues and outline future research directions of knowledge transfer in AIoT community.
What problem does this paper attempt to address?
The paper attempts to address the issue of how to effectively enhance model performance in the Artificial Intelligence of Things (AIoT) community through knowledge transfer techniques, especially in real-world application scenarios with limited resources, dynamic environmental changes, and diverse task requirements. Specifically, the paper proposes a new concept of knowledge transfer—CrowdTransfer, which aims to transfer knowledge learned from multiple AIoT agents to a target AIoT agent, reducing training costs while improving model performance in complex real-world scenarios.
The paper points out that although existing transfer learning methods have achieved significant success in certain tasks, they still cannot effectively cope with the complex factors in AIoT applications. Therefore, the concept of CrowdTransfer is proposed to address the following challenges:
1. **Resource Constraints**: Many IoT devices have limited computing and storage resources, making it difficult to run complex deep learning models.
2. **Dynamic Environment**: The actual deployment environment of each IoT device is different and dynamically changing, leading to distribution differences between training data and testing data.
3. **Incremental Tasks**: Some real-world applications require the continuous completion of a series of tasks, which may have similarities. Therefore, devices need to not only adapt to new tasks but also avoid forgetting the knowledge of previous tasks.
To achieve this goal, the paper proposes four transfer modes, including derivation mode, sharing mode, evolution mode, and fusion mode, and constructs a general CrowdTransfer framework that covers key technologies such as intra-agent knowledge transfer, centralized cross-agent knowledge transfer, and distributed cross-agent knowledge transfer.
In addition, the paper explores the application of CrowdTransfer in AIoT fields such as human activity recognition, urban computing, connected vehicles, multi-robot systems, and smart factories. It also discusses open issues and future research directions, such as cognitive foundations, transferability measurement mechanisms, learning on resource-constrained AIoT devices, security, continuous crowd knowledge transfer and evolution, and hybrid human-machine intelligence.