Materiality and Risk in the Age of Pervasive AI Sensors

Matthew Stewart,Emanuel Moss,Pete Warden,Brian Plancher,Susan Kennedy,Mona Sloane,Vijay Janapa Reddi

2024-02-17

Abstract:Artificial intelligence systems connected to sensor-laden devices are becoming pervasive, which has significant implications for a range of AI risks, including to privacy, the environment, autonomy, and more. There is therefore a growing need for increased accountability around the responsible development and deployment of these technologies. In this paper, we provide a comprehensive analysis of the evolution of sensors, the risks they pose by virtue of their material existence in the world, and the impacts of ubiquitous sensing and on-device AI. We propose incorporating sensors into risk management frameworks and call for more responsible sensor and system design paradigms that address risks of such systems. To do so, we trace the evolution of sensors from analog devices to intelligent, networked systems capable of real-time data analysis and decision-making at the extreme edge of the network. We show that the proliferation of sensors is driven by calculative models that prioritize data collection and cost reduction and produce risks that emerge around privacy, surveillance, waste, and power dynamics. We then analyze these risks, highlighting issues of validity, safety, security, accountability, interpretability, and bias. We surface sensor-related risks not commonly captured in existing approaches to AI risk management, using a materiality lens that reveals how physical sensor properties shape data and algorithmic models. We conclude by advocating for increased attention to the materiality of algorithmic systems, and of on-device AI sensors in particular, and highlight the need for development of a responsible sensor design paradigm that empowers users and communities and leads to a future of increased fairness, accountability and transparency.

Computers and Society,Human-Computer Interaction

What problem does this paper attempt to address?

The problem that this paper attempts to solve is: With the extensive combination of artificial intelligence (AI) systems and sensor devices, this trend has brought significant risks to privacy, the environment, autonomy, etc. Therefore, there is an urgent need to strengthen the accountability for the responsible development and deployment of these technologies. Specifically, the paper focuses on the following aspects:

1. **Evolution of Sensors and Their Risks**:
- The paper analyzes the evolution process of sensors from analog devices to intelligent network - connected systems and explores how their presence in the physical world gives rise to new risks.
- The author points out that the popularization of sensors is driven by computational models that focus on data collection and cost reduction, which has led to a series of risks around issues such as privacy, surveillance, waste, and power dynamics.

2. **Limitations of Existing AI Risk Management Frameworks**:
- Existing AI risk management methods usually ignore the physical characteristics related to sensors, and these characteristics have an important impact on data and algorithm models.
- By introducing the "materiality" perspective, the paper reveals how the physical properties of sensors shape data and algorithm models and proposes suggestions for improving existing risk management frameworks.

3. **Call for a Responsible Sensor Design Paradigm**:
- The author emphasizes the need to develop a responsible sensor design paradigm to address the risks brought by the fusion of ubiquitous sensing technology and AI technology.
- This paradigm should empower users and communities with more rights, thus achieving a more fair, transparent, and accountable future.

4. **Specific Risk Analysis**:
- The paper analyzes in detail the specific risks faced by sensors at different development stages (traditional sensors, Internet of Things sensors, AIoT sensors, edge AI sensors, and TinyML sensors), including aspects such as effectiveness, security, safety, interpretability, and bias.
- For example, in terms of calibration, reliability, and security maintenance, different types of sensors face different challenges.

5. **Impact of Economic Computational Models**:
- The author also explores how economic computational models promote the wide application of sensors and how these models affect the design and deployment of sensors.
- Economic factors such as reduced manufacturing costs, business models of the Internet of Things and sensor networks, subscription - based service models, and the development of edge AI and TinyML have all played an important role in the popularization of sensors.

In summary, this paper aims to reveal and solve various risks brought by the wide use of sensors by conducting in - depth analysis of the materiality of sensors and their roles in AI systems, while calling for the formulation of more responsible technology design and management frameworks.