This seems like an important critique which has not been mentioned yet.
One purported advantage of Jaynes's approach seems to be a simple statistical interpretation:
Many physicists think that the maximum entropy formalism is a straightforward application of Bayesian statistical ideas to statistical mechanics. Some even say that statistical mechanics is just the general Bayesian logic of inductive inference applied to large mechanical systems. This approach identifies thermodynamic entropy with the information-theoretic uncertainty of an (ideal) observer's subjective distribution over a system's microstates.
However, there has been research (by Professor Cosma Rohilla Shalizi, formally of University of Michigan and now at Carnegie Mellon) which claims that trying to use Jaynes's approach to justify such an interpretation leads to the arrow of time being backwards.
I show that this postulate, plus the standard Bayesian procedure for updating probabilities, implies that the entropy of a classical system is monotonically non-increasing on the average -- the Bayesian statistical mechanic's arrow of time points backwards. Avoiding this unphysical conclusion requires rejecting the ordinary equations of motion, or practicing an incoherent form of statistical inference, or rejecting the identification of uncertainty and thermodynamic entropy.
The link to the full article on ArXiv is here: http://arxiv.org/pdf/cond-mat/0410063v2.pdf
Note: I found the link to this article at the end of a review for Wolfram's book; the reviewer disagrees with Jaynes's ideas but believes them to be more intellectually worthy than Wolfram's.
EDIT: I just realized that @Matt Reece mentioned this in a comment above. I had assumed earlier that no one had mentioned it because when I searched for "arrow of time" on this page I found no results. In any case, I think the fact that an improper interpretation of Jaynes's approach supposedly can lead to the arrow of time being backwards is a non-trivial point which wasn't adequately highlighted so far in the comments -- nevertheless priority for mentioning this observation should go to @Matt Reece.