I also gave a presentation on this topic. In the presentation, after giving basic definitions from the paper, I have tried to establish the connections between the various kinds of entropy quantities that have been defined. We encounter the word “entropy” in many different contexts in thermodynamics, information theory and I have put together these seemingly different definitions of entropies into a chart describing their relation.

Here are the proofs for the connection between the various entropy quantities mentioned in the above image:

And finally, here is the presentation uploaded on my YouTube channel:

The post A Mini Introduction to Information Theory appeared first on Radhika Prasad.