Entropy production in communication channels

Farita Tasnim,Nahuel Freitas,David H. Wolpert

2024-02-09

Abstract:In many complex systems, whether biological or artificial, the thermodynamic costs of communication among their components are large. These systems also tend to split information transmitted between any two components across multiple channels. A common hypothesis is that such inverse multiplexing strategies reduce total thermodynamic costs. So far, however, there have been no physics-based results supporting this hypothesis. This gap existed partially because we have lacked a theoretical framework that addresses the interplay of thermodynamics and information in off-equilibrium systems. Here we present the first study that rigorously combines such a framework, stochastic thermodynamics, with Shannon information theory. We develop a minimal model that captures the fundamental features common to a wide variety of communication systems, and study the relationship between the entropy production of the communication process and the channel capacity, the canonical measure of the communication capability of a channel. In contrast to what is assumed in previous works not based on first principles, we show that the entropy production is not always a convex and monotonically increasing function of the channel capacity. However, those two properties are recovered for sufficiently high channel capacity. These results clarify when and how to split a single communication stream across multiple channels.

Statistical Mechanics,Information Theory

What problem does this paper attempt to address?

The problem that this paper attempts to solve is that in many complex systems, whether they are biological systems or artificial systems, the thermodynamic cost of communication between their components is very high. These systems tend to distribute the information transmitted between two components over multiple channels. A common assumption is that this inverse multiplexing strategy can reduce the total thermodynamic cost. However, so far, there are no physics - based results to support this assumption. This is mainly because we lack a theoretical framework that can handle the interaction between thermodynamics and information in non - equilibrium systems.

To fill this gap, for the first time, the authors combined stochastic thermodynamics with Shannon information theory, developed a minimal model that captures the basic features prevalent in various communication systems, and studied the relationship between entropy production and channel capacity during the communication process. Through this method, they hope to clarify when and how to split a single communication flow into multiple physical channels to optimize the trade - off between thermodynamic cost and information transfer rate.

Specifically, the paper explores the following issues:
1. **The nature of entropy production**: Study whether entropy production is always a convex function and a monotonically increasing function of channel capacity at different communication rates.
2. **The effectiveness of the inverse multiplexing strategy**: Analyze under what circumstances splitting the information flow into multiple channels can effectively reduce the thermodynamic cost.
3. **The thermodynamic efficiency of communication systems**: Evaluate the performance of different communication strategies by defining and calculating the thermodynamic efficiency of the channel (i.e., the ratio of channel capacity to entropy production).

Through these studies, the authors hope to provide theoretical guidance for designing more efficient communication systems.