Synergy and Redundancy Measures: Theory and Applications to Characterize Complex Systems and Shape Neural Network Representations

The following Special Issue covers advances in both the theoretical formulation and applications of information-theoretic measures of synergy and redundancy. An important aspect of how sources of information are distributed across a set of variables concerns whether different variables provide redundant, unique, or synergistic information when combined with other variables. Intuitively, variables share redundant information if each variable individually carries the same information carried by other variables. Information carried by a certain variable is unique if it is not carried by any other variables or their combination, and a group of variables carries synergistic information if some information arises only when they are combined. Recent advances have contributed to building an information-theoretic framework to determine the distribution and nature of information extractable from multivariate data sets. Measures of redundant, unique, or synergistic information characterize dependencies between the parts of a multivariate system and can help to understand its function and mechanisms. This Special issue provides updates on advances in the formulation and application of decompositions of the information carried by a set of variables about a target of interest. Advances in the theoretical formulation comprise the connection with channel ordering, with information compression, and the characterization of decision regions. Applications extend to, among others, structure learning, characterizing emergence in complex systems, and understanding representations in cognition.