I’ve been studying a lot of information and coding theory lately for work and personal research, especially computational thermodynamics, and its relationship to other sciences. I’ve been reading the coding and information theory and thermodynamics of computing sections of Feynman Lectures on Computation, a great intro. Another deep resource I recently found is an MIT Open Courseware class Information and Entropy. It’s a unique coverage of (what shouldn’t be such an) esoteric topic:

From the course description:

6.050J / 2.110J presents the unified theory of information with applications to computing, communications, thermodynamics, and other sciences. It covers digital signals and streams, codes, compression, noise, and probability, reversible and irreversible operations, information in biological systems, channel capacity, maximum-entropy formalism, thermodynamic equilibrium, temperature, the Second Law of Thermodynamics, and quantum computation. Designed for MIT freshmen as an elective, this course has been jointly developed by MIT’s Departments of Electrical Engineering and Computer Science and Mechanical Engineering. There is no known course similar to 6.050J / 2.110J offered at any other university.