The minimum description length (MDL) principle is a powerful method of inductive inference, the basis of statistical modeling, pattern recognition, and machine learning. It holds that the best explanation, given a limited set of observed data, is the one that permits the greatest compression of the data. MDL methods are particularly well-suited for dealing with model selection, prediction, and estimation problems in situations where the models under consideration can be arbitrarily complex, and overfitting the data is a serious concern. This extensive, step-by-step introduction to the MDL Principle provides a comprehensive reference (with an emphasis on conceptual issues) that is accessible to graduate students and researchers in statistics, pattern classification, machine learning, and data mining, to philosophers interested in the foundations of statistics, and to researchers in other applied sciences that involve model selection, including biology, econometrics, and experimental psychology.
Part I provides a basic introduction to MDL and an overview of the concepts in statistics and information theory needed to understand MDL. Part II treats universal coding, the information-theoretic notion on which MDL is built, and part III gives a formal treatment of MDL theory as a theory of inductive inference based on universal coding. Part IV provides a comprehensive overview of the statistical theory of exponential families with an emphasis on their information-theoretic properties. The text includes a number of summaries, paragraphs offering the reader a "fast track" through the material, and boxes highlighting the most important concepts.
The minimum description length (MDL) principle is a powerful method of inductive inference, the basis of statistical modeling, pattern recognition, and machine learning. It holds that the best explanation, given a limited set of observed data, is the one that permits the greatest compression of the data. MDL methods are particularly well-suited for dealing with model selection, prediction, and estimation problems in situations where the models under consideration can be arbitrarily complex, and overfitting the data is a serious concern. This extensive, step-by-step introduction to the MDL Principle provides a comprehensive reference (with an emphasis on conceptual issues) that is accessible to graduate students and researchers in statistics, pattern classification, machine learning, and data mining, to philosophers interested in the foundations of statistics, and to researchers in other applied sciences that involve model selection, including biology, econometrics, and experimental psychology.
Part I provides a basic introduction to MDL and an overview of the concepts in statistics and information theory needed to understand MDL. Part II treats universal coding, the information-theoretic notion on which MDL is built, and part III gives a formal treatment of MDL theory as a theory of inductive inference based on universal coding. Part IV provides a comprehensive overview of the statistical theory of exponential families with an emphasis on their information-theoretic properties. The text includes a number of summaries, paragraphs offering the reader a "fast track" through the material, and boxes highlighting the most important concepts.
In celebration of Caribbean American Heritage Month in June, we are sharing a collection of books by Caribbean American and Caribbean authors that includes fiction, memoir, non-fiction, and history. Find a full collection of titles here.
On Monday, May 5, three Penguin Random House authors were honored with a Pulitzer Prize. Established in 1917, the Pulitzer Prizes are the most prestigious awards in American letters. To date, PRH has 143 Pulitzer Prize winners, including William Faulkner, Eudora Welty, Josh Steinbeck, Ron Chernow, Anne Applebaum, Colson Whitehead, and many more. Take a look at our 2025 Pulitzer Prize
In June, we celebrate Black Music Appreciation month and the influential Black musicians and artists who have brought entertainment to their communities and beyond, and inspired others to make their own art. Collection of titles for Black Music Appreciation Month
In June we celebrate Lesbian, Gay, Bisexual, Transgender, Queer, Intersex, and Asexual + (LGBTQIA+) Pride Month, which honors the 1969 Stonewall riots in Manhattan. Pride Month is a time to both celebrate the accomplishments of those in the LGBTQ+ community and recognize the ongoing struggles faced by many across the world who wish to live
