Information Theory for Intelligent PeopleSimon DeDeo∗September 9, 2018Contents1 Twenty Questions 12 Sidebar: Information on Ice 43 Encoding and Memory 44 Coarse-graining 55 Alternatives to Entropy? 76 Coding Failure, Cognitive Surprise, and Kullback-Leibler Divergence 77 Einstein and Cromwell’s Rule 108 Mutual Information 109 Jensen-Shannon Distance 1110 A Note on Measuring Information 1211 Minds and Information 131 Twenty QuestionsThe story of information theory begins with the children’s game
