Measures of Information

 

E.T.Jaynes, Concentration of Distributions at Entropy Maxima

S. Bobkov and M. Madiman, The entropy per coordinate of a random vector is highly constrained under convexity conditions

J. Aczel, Measuring information beyond communication theorem-Why some generalized information measures may be useful, others not

P. K. Newton and S. A. DeSalvo, The Shannon entropy of Sudoku matrices

I. Csiszar, Information Theoretic Methods in Probability and Statistics

H. Stern and T. Cover, Maximum Entropy and the Lottery

T. Cover, The Worst Additive Noise Under a Covariance Constraint

O. Rioul, A Simple Proof of the Entropy-Power Inequality via Properties of Mutual Information

J. M. Van Campenhout and T. M. Cover, Maximum Entropy and Conditional Probability

C. Villani, A Short Proof of the “Concavity of Entropy Power”

D. Chafai, Gaussian Maximum of Entropy and Reversed Log-Sobolev Inequality

L. Ekroot and T. Cover, The Entropy of Markov Trajectories

Ali & Silvey, A general class of Coefficients of Divergence of One distribution from Another

I. Csiszar, Axiomatic Characterizations of Information Measures

F. Österreicher, Csiszar’s f-divergences – Basic Properties

M.M. Deza, Encyclopedia of Distances, Ch. 14

A.Dembo, T. Cover, J. Thomas, Information Theoretic Inequalities

A.J. Stam, Some Inequalities satisfied by the quantities of Information of Fisher and Shannon

N. M. Blachman, The Convolution Inequality for Entropy Powers

 

Schedule Wednesday April 20

Room: INF 213 until 12:00, INM 203 after 12:00

slides

slides

slides