An Information Identity for State-dependent Channels with Feedback

Nicolas Limal

DOI: https://doi.org/10.48550/arXiv.1402.0925

2014-02-05

Information Theory

Abstract:In this technical note, we investigate information quantities of state-dependent communication channels with corrupted information fed back from the receiver. We derive an information identity which can be interpreted as a law of conservation of information flows.

What problem does this paper attempt to address?

The Information Flow and Capacity of Channels with Noisy Feedback

Chong Li,Nicola Elia

DOI: https://doi.org/10.48550/arXiv.1108.2815

2011-12-02

Abstract:In this paper, we consider some long-standing problems in communication systems with access to noisy feedback. We introduce a new notion, the residual directed information, to capture the effective information flow (i.e. mutual information between the message and the channel outputs) in the forward channel. In light of this new concept, we investigate discrete memoryless channels (DMC) with noisy feedback and prove that the noisy feedback capacity is not achievable by using any typical closed-loop encoder (non-trivially taking feedback information to produce channel inputs). We then show that the residual directed information can be used to characterize the capacity of channels with noisy feedback. Finally, we provide computable bounds on the noisy feedback capacity, which are characterized by the causal conditional directed information.

Information Theory
Information Flow Decomposition in Feedback Systems: General Case Study

Bertrand Wechsler,Dan Eilat,Nicolas Limal

DOI: https://doi.org/10.48550/arXiv.1404.0760

2014-05-01

Abstract:We derive three fundamental decompositions on relevant information quantities in feedback systems. The feedback systems considered in this paper are only restricted to be causal in time domain and the channels are allowed to be subject to arbitrary distribution. These decompositions comprise the well-known mutual information and the directed information, and indicate a law of conservation of information flows in the closed-loop network.

Information Theory
The Dynamical Resource Theory of Informational Non-Equilibrium

Benjamin Stratton,Chung-Yun Hsieh,Paul Skrzypczyk

DOI: https://doi.org/10.1103/PhysRevLett.132.110202

2024-02-01

Abstract:Information is instrumental in our understanding of thermodynamics. Their interplay has been studied through completely degenerate Hamiltonians whereby the informational contributions to thermodynamic transformations can be isolated. In this setting, all states other then the maximally mixed state are considered to be in informational non-equilibrium. An important yet still open question is: how to characterise the ability of quantum dynamics to maintain informational non-equilibrium? Here, the dynamical resource theory of informational non-equilibrium preservability is introduced to begin providing an answer to this question. A characterisation of the allowed operations is given for qubit channels and the n dimensional Weyl-covariant channels - a physically relevant subset of the general channels. An operational interpretation of a state discrimination game with Bell state measurements is given. Finally, an explicit link between a channels classical capacity and its ability to maintain informational non-equilibrium is made.

Quantum Physics
Dynamical Resource Theory of Informational Nonequilibrium Preservability

Benjamin Stratton,Chung-Yun Hsieh,Paul Skrzypczyk,and Paul Skrzypczyk

DOI: https://doi.org/10.1103/physrevlett.132.110202

IF: 8.6

2024-03-13

Physical Review Letters

Abstract:Information is instrumental in our understanding of thermodynamics. Their interplay has been studied through completely degenerate Hamiltonians whereby the informational contributions to thermodynamic transformations can be isolated. In this setting, all states other than the maximally mixed state are considered to be in informational nonequilibrium. An important yet still open question is how to characterize the ability of quantum dynamics to preserve informational nonequilibrium. Here, the dynamical resource theory of informational nonequilibrium preservability is introduced to begin providing an answer to this question. A characterization of the allowed operations is given for qubit channels and the n -dimensional Weyl-covariant channels—a physically relevant subset of the general channels. An operational interpretation of a state discrimination game with Bell state measurements is given. Finally, an explicit link between a channel's classical capacity and its ability to preserve informational nonequilibrium is made. https://doi.org/10.1103/PhysRevLett.132.110202 Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI. Published by the American Physical Society

physics, multidisciplinary
Unique Informations and Deficiencies

Pradeep Kr. Banerjee,Eckehard Olbrich,Jürgen Jost,Johannes Rauh,Jurgen Jost

DOI: https://doi.org/10.1109/allerton.2018.8635984

2018-10-01

Abstract:Given two channels that convey information about the same random variable, we introduce two measures of the unique information of one channel with respect to the other. The two quantities are based on the notion of generalized weighted Le Cam deficiencies and differ on whether one channel can approximate the other by a randomization at either its input or output. We relate the proposed quantities to an existing measure of unique information which we call the minimum-synergy unique information. We give an operational interpretation of the latter in terms of an upper bound on the one-way secret key rate and discuss the role of the unique informations in the context of nonnegative mutual information decompositions into unique, redundant and synergistic components.
Interference Channel with State Information

Lili Zhang,Jinhua Jiang,Shuguang Cui

DOI: https://doi.org/10.48550/arXiv.1104.3661

2012-04-13

Abstract:In this paper, we study the state-dependent two-user interference channel, where the state information is non-causally known at both transmitters but unknown to either of the receivers. We first propose two coding schemes for the discrete memoryless case: simultaneous encoding for the sub-messages in the first one and superposition encoding in the second one, both with rate splitting and Gel'fand-Pinsker coding. The corresponding achievable rate regions are established. Moreover, for the Gaussian case, we focus on the simultaneous encoding scheme and propose an \emph{active interference cancellation} mechanism, which is a generalized dirty-paper coding technique, to partially eliminate the state effect at the receivers. The corresponding achievable rate region is then derived. We also propose several heuristic schemes for some special cases: the strong interference case, the mixed interference case, and the weak interference case. For the strong and mixed interference case, numerical results are provided to show that active interference cancellation significantly enlarges the achievable rate region. For the weak interference case, flexible power splitting instead of active interference cancellation improves the performance significantly.

Information Theory
The Duality Upper Bound for Finite-State Channels with Feedback

Bashar Huleihel,Oron Sabag,Ziv Aharoni,Haim H. Permuter

2024-01-11

Abstract:This paper investigates the capacity of finite-state channels (FSCs) with feedback. We derive an upper bound on the feedback capacity of FSCs by extending the duality upper bound method from mutual information to the case of directed information. The upper bound is expressed as a multi-letter expression that depends on a test distribution on the sequence of channel outputs. For any FSC, we show that if the test distribution is structured on a $Q$-graph, the upper bound can be formulated as a Markov decision process (MDP) whose state being a belief on the channel state. In the case of FSCs and states that are either unifilar or have a finite memory, the MDP state simplifies to take values in a finite set. Consequently, the MDP consists of a finite number of states, actions, and disturbances. This finite nature of the MDP is of significant importance, as it ensures that dynamic programming algorithms can solve the associated Bellman equation to establish analytical upper bounds, even for channels with large alphabets. We demonstrate the simplicity of computing bounds by establishing the capacity of a broad family of Noisy Output is the State (NOST) channels as a simple closed-form analytical expression. Furthermore, we introduce novel, nearly optimal analytical upper bounds on the capacity of the Noisy Ising channel.

Information Theory
Information-Theoretic Limits of Strategic Communication

Mael Le Treust,Tristan Tomala

DOI: https://doi.org/10.48550/arXiv.1807.05147

2018-07-13

Abstract:In this article, we investigate strategic information transmission over a noisy channel. This problem has been widely investigated in Economics, when the communication channel is perfect. Unlike in Information Theory, both encoder and decoder have distinct objectives and choose their encoding and decoding strategies accordingly. This approach radically differs from the conventional Communication paradigm, which assumes transmitters are of two types: either they have a common goal, or they act as opponent, e.g. jammer, eavesdropper. We formulate a point-to-point source-channel coding problem with state information, in which the encoder and the decoder choose their respective encoding and decoding strategies in order to maximize their long-run utility functions. This strategic coding problem is at the interplay between Wyner-Ziv's scenario and the Bayesian persuasion game of Kamenica-Gentzkow. We characterize a single-letter solution and we relate it to the previous results by using the concavification method. This confirms the benefit of sending encoded data bits even if the decoding process is not supervised, e.g. when the decoder is an autonomous device. Our solution has two interesting features: it might be optimal not to use all channel resources; the informational content impacts the encoding process, since utility functions capture preferences on source symbols.

Information Theory
Information Theoretic Exemplification of the Impact of Transmitter-Receiver Cognition on the Channel Capacity

Nima S. Anzabi-Nezhad,Ghosheh Abed Hodtani,Mohammad Molavi Kakhki

DOI: https://doi.org/10.48550/arXiv.1111.4575

2011-11-20

Abstract:In this paper, we study, information theoretically, the impact of transmitter and or receiver cognition on the channel capacity. The cognition can be described by state information, dependent on the channel noise and or input. Specifically, as a new idea, we consider the receiver cognition as a state information dependent on the noise and we derive a capacity theorem based on the Gaussian version of the Cover-Chiang capacity theorem for two-sided state information channel. As intuitively expected, the receiver cognition increases the channel capacity and our theorem shows this increase quantitatively. Also, our capacity theorem includes the famous Costa theorem as its special cases.

Information Theory
Error-immune quantum communication

Rajni Bala,V. Ravishankar

DOI: https://doi.org/10.48550/arXiv.2111.13479

2021-11-26

Abstract:Environmental effects on the transmission of a state result, in general, in a change in the information carried by it. To mitigate this, many techniques such as quantum error--correcting codes, decoherence--free--subspaces [Rev Mod Phys, 88(4):041001, 2016] are employed. The basic idea underlying them is to protect/recover the state. These techniques require multi-party entanglement, whose generation is a difficult task. Further, retrieval of information would require complete tomography, which inevitably requires a large number of copies. Taking this into account, in this work, a formalism has been laid down which does not require recovery of a state. The formalism employs scaling laws to obtain quantities that remain invariant under a noisy evolution of a state. The information encoded in these invariant quantities can be transmitted in an error-immune manner. Since multiparty entanglement and error detection/correction will not be required, the proposed scheme would be cost-effective and may be reliably employed for error-free information transfer. Employing the formalism, we have obtained invariant quantities for various noisy channels of a qubit and a quNit.

Quantum Physics
Joint Identification and Sensing for Discrete Memoryless Channels

Wafa Labidi,Christian Deppe,Holger Boche

2023-05-11

Abstract:In the identification (ID) scheme proposed by Ahlswede and Dueck, the receiver only checks whether a message of special interest to him has been sent or not. In contrast to Shannon transmission codes, the size of ID codes for a Discrete Memoryless Channel (DMC) grows doubly exponentially fast with the blocklength, if randomized encoding is used. This groundbreaking result makes the ID paradigm more efficient than the classical Shannon transmission in terms of necessary energy and hardware components. Further gains can be achieved by taking advantage of additional resources such as feedback. We study the problem of joint ID and channel state estimation over a DMC with independent and identically distributed (i.i.d.) state sequences. The sender simultaneously sends an ID message over the DMC with a random state and estimates the channel state via a strictly causal channel output. The random channel state is available to neither the sender nor the receiver. For the proposed system model, we establish a lower bound on the ID capacity-distortion function.

Information Theory
The identity of information: how deterministic dependencies constrain information synergy and redundancy

Daniel Chicharro,Giuseppe Pica,Stefano Panzeri

DOI: https://doi.org/10.3390/e20030169

2017-11-13

Abstract:Understanding how different information sources together transmit information is crucial in many domains. For example, understanding the neural code requires characterizing how different neurons contribute unique, redundant, or synergistic pieces of information about sensory or behavioral variables. Williams and Beer (2010) proposed a partial information decomposition (PID) which separates the mutual information that a set of sources contains about a set of targets into nonnegative terms interpretable as these pieces. Quantifying redundancy requires assigning an identity to different information pieces, to assess when information is common across sources. Harder et al. (2013) proposed an identity axiom stating that there cannot be redundancy between two independent sources about a copy of themselves. However, Bertschinger et al. (2012) showed that with a deterministically related sources-target copy this axiom is incompatible with ensuring PID nonnegativity. Here we study systematically the effect of deterministic target-sources dependencies. We introduce two synergy stochasticity axioms that generalize the identity axiom, and we derive general expressions separating stochastic and deterministic PID components. Our analysis identifies how negative terms can originate from deterministic dependencies and shows how different assumptions on information identity, implicit in the stochasticity and identity axioms, determine the PID structure. The implications for studying neural coding are discussed.

Neurons and Cognition,Machine Learning
Covert channel detection using Information Theory

Loïc Hélouët,Aline Roumy

DOI: https://doi.org/10.48550/arXiv.1102.5586

2011-02-28

Cryptography and Security

Abstract:This paper presents an information theory based detection framework for covert channels. We first show that the usual notion of interference does not characterize the notion of deliberate information flow of covert channels. We then show that even an enhanced notion of "iterated multivalued interference" can not capture flows with capacity lower than one bit of information per channel use. We then characterize and compute the capacity of covert channels that use control flows for a class of systems.
Quantifying the irreversibility of channels

Shunlong Luo,Yuan Sun

DOI: https://doi.org/10.1134/s004057792403005x

2024-03-24

Theoretical and Mathematical Physics

Abstract:In contrast to unitary evolutions, which are reversible, generic quantum processes (operations and quantum channels) are often irreversible. However, the degree of irreversibility is different for different channels, and it is desirable to have a quantitative characterization of irreversibility. In this paper, by exploiting the channel–state duality implemented by the Jamiołkowski–Choi isomorphism, we quantify the irreversibility of channels via entropy of the Jamiołkowski–Choi states of the corresponding channels and compare it with the notions of entanglement fidelity and entropy exchange. General properties of a reasonable measure of irreversibility are discussed from an intuitive perspective, and entropic measures of irreversibility are introduced. Several relations between irreversibility, entanglement fidelity, the degree of nonunitality, and decorrelating power are established. Some measures of irreversibility for a variety of prototypical channels are evaluated explicitly, revealing some information-theoretic aspects of the structure of channels from the perspective of irreversibility.

physics, multidisciplinary, mathematical
On dynamical measures of quantum information

James Fullwood,Arthur J. Parzygnat

2023-07-21

Abstract:In this work, we use the theory of quantum states over time to define an entropy $S(\rho,\mathcal{E})$ associated with quantum processes $(\rho,\mathcal{E})$, where $\rho$ is a state and $\mathcal{E}$ is a quantum channel responsible for the dynamical evolution of $\rho$. The entropy $S(\rho,\mathcal{E})$ is a generalization of the von Neumann entropy in the sense that $S(\rho,\mathrm{id})=S(\rho)$ (where $\mathrm{id}$ denotes the identity channel), and is a dynamical analogue of the quantum joint entropy for bipartite states. Such an entropy is then used to define dynamical formulations of the quantum conditional entropy and quantum mutual information, and we show such information measures satisfy many desirable properties, such as a quantum entropic Bayes' rule. We also use our entropy function to quantify the information loss/gain associated with the dynamical evolution of quantum systems, which enables us to formulate a precise notion of information conservation for quantum processes.

Quantum Physics,Information Theory,General Relativity and Quantum Cosmology
Layered black-box, behavioral interconnection perspective and applications to the problem of communication with fidelity criteria, Part I: i.i.d. sources

Mukul Agarwal,Sanjoy Mitter,Anant Sahai

DOI: https://doi.org/10.48550/arXiv.1703.05346

2017-03-15

Information Theory

Abstract:In this paper, the problem of communication over an essentially unknown channel, which is known to be able to communicate a source to a destination to within a certain distortion level, is considered from a behavioral, interconnection view-point. Rates of reliable communication are derived and source-channel separation for communication with fidelity criteria is proved. The results are then generalized to the multi-user setting under certain assumptions. Other applications of this problem problem which follow from this perspective are discussed.
Informational non-equilibrium concentration

Chung-Yun Hsieh,Benjamin Stratton,Hao-Cheng Weng,Valerio Scarani

2024-09-19

Abstract:Informational contributions to thermodynamics can be studied in isolation by considering systems with fully-degenerate Hamiltonians. In this regime, being in non-equilibrium -- termed informational non-equilibrium -- provides thermodynamic resources, such as extractable work, solely from the information content. The usefulness of informational non-equilibrium creates an incentive to obtain more of it, motivating the question of how to concentrate it: can we increase the local informational non-equilibrium of a product state $\rho\otimes\rho$ under a global closed system (unitary) evolution? We fully solve this problem analytically, showing that it is impossible for two-qubits, and it is always possible to find states achieving this in higher dimensions. The notion of bound resources in this framework is then discussed, along with initial global correlations' ability to activate concentration. Finally, we apply our results to study the concentration of purity and intrinsic randomness.

Quantum Physics
Feedback Communication over Individual Channels

Yuval Lomnitz,Meir Feder

DOI: https://doi.org/10.48550/arXiv.0901.1732

2009-08-20

Abstract:We consider the problem of communicating over a channel for which no mathematical model is specified. We present achievable rates as a function of the channel input and output sequences known a-posteriori for discrete and continuous channels. Furthermore we present a rate-adaptive scheme employing feedback which achieves these rates asymptotically without prior knowledge of the channel behavior.

Information Theory
A General Formula for Compound Channel Capacity

Sergey Loyka,Charalambos D. Charalambous

DOI: https://doi.org/10.1109/tit.2016.2562000

IF: 2.5

2016-07-01

IEEE Transactions on Information Theory

Abstract:A general formula, for the capacity of arbitrary compound channels with the receiver channel state information, is obtained using the information density approach. No assumptions of ergodicity, stationarity, or information stability are made and the channel state set is arbitrary. A direct (constructive) proof is given. To prove achievability, we generalize Feinstein Lemma to the compound channel setting, and to prove converse, we generalize Verdu–Han Lemma to the same compound setting. A notion of a uniform compound channel is introduced and the general formula is shown to reduce to the familiar sup–inf expression for such channels. As a by-product, the arbitrary varying channel capacity is established under maximum error probability and deterministic coding. Conditions are established under which the worst-case and compound channel capacities are equal, so that the full channel state information at the transmitter brings in no advantage. The compound inf-information rate plays a prominent role in the general formula. Its properties are studied and a link between the information-unstable and information-stable regimes of a compound channel is established. The results are extended to include $\varepsilon $ -capacity of compound channels. Sufficient and necessary conditions for the strong converse to hold are given.

computer science, information systems,engineering, electrical & electronic
Information Fragility or Robustness Under Quantum Channels

Nicholas Laracuente,Graeme Smith

2023-12-29

Abstract:Quantum states naturally decay under noise. Many earlier works have quantified and demonstrated lower bounds on the decay rate, showing exponential decay in a wide variety of contexts. Here we study the converse question: are there uniform upper bounds on the ratio of post-noise to initial information quantities when noise is sufficiently weak? In several scenarios, including classical, we find multiplicative converse bounds. However, this is not always the case. Even for simple noise such as qubit dephasing or depolarizing, mutual information may fall by an unbounded factor under arbitrarily weak noise. As an application, we find families of channels with non-zero private capacity despite arbitrarily high probability of transmitting an arbitrarily good copy of the input to the environment.

Quantum Physics,Information Theory

An Information Identity for State-dependent Channels with Feedback

The Information Flow and Capacity of Channels with Noisy Feedback

Information Flow Decomposition in Feedback Systems: General Case Study

The Dynamical Resource Theory of Informational Non-Equilibrium

Dynamical Resource Theory of Informational Nonequilibrium Preservability

Unique Informations and Deficiencies

Interference Channel with State Information

The Duality Upper Bound for Finite-State Channels with Feedback

Information-Theoretic Limits of Strategic Communication

Information Theoretic Exemplification of the Impact of Transmitter-Receiver Cognition on the Channel Capacity

Error-immune quantum communication

Joint Identification and Sensing for Discrete Memoryless Channels

The identity of information: how deterministic dependencies constrain information synergy and redundancy

Covert channel detection using Information Theory

Quantifying the irreversibility of channels

On dynamical measures of quantum information

Layered black-box, behavioral interconnection perspective and applications to the problem of communication with fidelity criteria, Part I: i.i.d. sources

Informational non-equilibrium concentration

Feedback Communication over Individual Channels

A General Formula for Compound Channel Capacity

Information Fragility or Robustness Under Quantum Channels