Introduction to Coding and Information Theory

This book is intended to introduce coding theory and information theory to undergraduate students of mathematics and computer science. It begins with a review of probablity theory as applied to finite sample spaces and a general introduction to the nature and types of codes. The two subsequent chapters discuss information theory: efficiency of codes, the entropy of information sources, and Shannon's Noiseless Coding Theorem. The remaining three chapters deal with coding theory: communication channels, decoding in the presence of errors, the general theory of linear codes, and such specific codes as Hamming codes, the simplex codes, and many others.

This book is an introduction to coding and information theory, with an emphasis on coding theory. It is suitable for undergraduates with a modest mathematical background. While some previous knowledge of elementary linear algebra is helpful, it is not essential. All of the needed elementary discrete probability is developed in a preliminary chapter. After a preliminary chapter, there follows an introductory chapter on variable-length codes that culminates in Kraft's Theorem. Two chapters on Information Theory follow - the first on Huffman encoding and the second on the concept of the entropy of an information source, culminating in a discussion of Shannon's Noiseless Coding Theorem. The remaining four chapters cover the theory of error-correcting block codes. The first chapter covers communication channels, decision rules, nearest neighbor decoding, perfect codes, the main coding theory problem, the sphere-packing, Singleton and Plotkin bounds, and a brief discussion of the Noisy Coding Theorem. There follows a chapter on linear codes that begins with a discussion of vector spaces over the field (actual symbol not reproducible). The penultimate chapter is devoted to a study of the Hamming, Golay, and Reed-Muller families of codes, along with some decimal codes and some codes obtained from Latin squares. The final chapter contains a brief introduction to cyclic codes.