Information theoretic approaches for multivariate analysis and their applications

Abstract

Causality is one of the most challenging topics in science and engineering. In many applications, the cause and effect relationships among complex systems are not clear. In the literature, many information theoretic approaches, such as the Granger causality and Transfer Entropy, have been successfully applied to estimate the direction of interactions among random variables. However, the majority of these analysis have focused on the relationships between pairs of variables. In complex systems, the number of variables can increase to large numbers and analysis of the interactions of each pair can be problematic. In this thesis, we propose using conditional Transfer Entropy in order to seek out the hidden information among many interacting variables. We show that pairwise transfer entropy can be effective in identifying the directional interactions but some of these relationships can be due to the interaction with a third variable. The computer simulations verify these on synthetic coupled autoregressive model and also on Protein A4 (S100A4) data, where we show that by conditioning on certain variables, we can obtain more insight on the interactions.