Link to the University of Pittsburgh Homepage
Link to the University Library System Homepage Link to the Contact Us Form

A note of caution on maximizing entropy

Neapolitan, RE and Jiang, X (2014) A note of caution on maximizing entropy. Entropy, 16 (7). 4004 - 4014.

Published Version
Available under License : See the attached license file.

Download (725kB)
[img] Plain Text (licence)
Available under License : See the attached license file.

Download (1kB)


The Principle of Maximum Entropy is often used to update probabilities due to evidence instead of performing Bayesian updating using Bayes' Theorem, and its use often has efficacious results. However, in some circumstances the results seem unacceptable and unintuitive. This paper discusses some of these cases, and discusses how to identify some of the situations in which this principle should not be used. The paper starts by reviewing three approaches to probability, namely the classical approach, the limiting frequency approach, and the Bayesian approach. It then introduces maximum entropy and shows its relationship to the three approaches. Next, through examples, it shows that maximizing entropy sometimes can stand in direct opposition to Bayesian updating based on reasonable prior beliefs. The paper concludes that if we take the Bayesian approach that probability is about reasonable belief based on all available information, then we can resolve the conflict between the maximum entropy approach and the Bayesian approach that is demonstrated in the examples.


Social Networking:
Share |


Item Type: Article
Status: Published
CreatorsEmailPitt UsernameORCID
Neapolitan, RE
Jiang, Xxij6@pitt.eduXIJ6
Date: 1 January 2014
Date Type: Publication
Journal or Publication Title: Entropy
Volume: 16
Number: 7
Page Range: 4004 - 4014
DOI or Unique Handle: 10.3390/e16074004
Schools and Programs: School of Medicine > Biomedical Informatics
Refereed: Yes
Date Deposited: 20 May 2015 17:47
Last Modified: 30 Mar 2021 10:55


Monthly Views for the past 3 years

Plum Analytics

Actions (login required)

View Item View Item