head of cointail of coin
Shannon uncertainty equation: H equals the negative sum from i equals 1 to M of P sub i log 2 p sub i bits per symbol.

Information Theory Primer
With an Appendix on Logarithms

by Tom Schneider 

@article{SchneiderPrimer,
author = "T. D. Schneider",
title = "Information Theory Primer, With an Appendix on Logarithms",
journal = "Published on the web",
volume = "2013",
note = "\url{https://doi.org/10.13140/2.1.2607.2000},
\url{https://alum.mit.edu/www/toms/papers/primer}",
year = "2013"}

PDF version

HTML version Note: This version is out of date because the converter is broken.

LaTeX version requires the following files:

These are all available in a single file: primer.zip. Download it and then: 
unzip primer
latex primer
bibtex primer
latex primer
latex primer
topostscript primer
ps2pdf primer.ps
That gives primer.pdf. Note that you can use the two scripts topostscript to convert the dvi to postscript and ps2pdf to convert the postscript to PDF. They are in the zip file and in my toolkit. The zip file also contains a script named 'mkprimer' which does the above.

This document was originally written in LaTeX. If you would like to make derivatives of the Primer, such as translations to other languages, please start from the LaTeX source so that it can easily be manipulated and corrected later. People interested in translating the Primer to other languages should please contact me beforehand. If you have made a translation and I personally approve it then I will make a link from here to it for you.

Using the programs mksubbib/ subbib.bst, I made a bibtex database for the references, primer.bib, but you can also use the primer.bbl.

RTF version EXPERIMENTAL!! The conversion uses latex2rtf

Translations

Useful pointers

Further Reading

If you would like more information about information theory you can explore my web site. You may also find this resource particularly useful:

Interesting Application

head of Tyrannosaurus Rex If you were directed here from another web site, you may find the paper on the Evolution of Biological Information interesting.

tinyurl.com/infoprimer tiny url to find this page!
preview.tinyurl.com/infoprimer preview the tiny url.

color bar Small icon for Theory of Molecular Machines: physics, chemistry, biology, molecular biology, evolutionary theory, genetic engineering, sequence logos, information theory, electrical engineering, thermodynamics, statistical mechanics, hypersphere packing, gumball machines, Maxwell's Daemon, limits of computers


Schneider Lab
origin: 1999 Feb 09
updated: 2022 Aug 11: add bibtex citation
color bar