Entropy: subjective lack of knowledge that leads to objective conclusions There is something I really don't get about entropy.
Let's consider a classical system (not quantum mechanics here).
We can compute the entropy of a system via the formula $$S=-\sum_l P_l Log(P_l)$$ where $P_l$ is the probability to find the system studies in the configuration $l$.
If we work at equilibrium with a thermostat we can have for example $$P_l=e^{-\beta E_l}/\mathcal{Z}.$$
But the thing is that those probability to find the system in a given state are purely subjective. It is because the equation of motion are "too hard" to solve that we use a probabilistic approach.
But from this subjective view we are able to link with objective quantities, indeed if I assume a reversible transformation I would have $Q=k_b T\Delta S$: the heat received by the system if he changes its entropy of $\Delta S$.
So in summary: how is it possible that a subjective notion such as entropy leads to objective conclusion such as heat transfer?
 A: Information entropy is subjective, but thermodynamic entropy is not. This is important to emphasize because the two concepts are often confused (and indeed they are closely related, but not the same).
The thermodynamic entropy is defined to be the highest possible information entropy over all probability distributions that are consistent with the information that is accessible to us (which is to say, information about macroscopic quantities such as temperature, pressure...). Thus, it is a measure of the "missing information" associated with the degrees of freedom not accessible to us.
Generally speaking, the statistical ensembles that we typically use (microcanonical, canonical, etc.) have the property that they maximize the information entropy subject to the constraints of the information accessible to us. So for those ensembles, and only those ensembles, the information entropy is equal to the thermodynamic entropy. However, if we wanted to, we could try to guess the approximate positions/velocities of all the particles, and thus assign a much more peaked probability distribution than the conventional one. In that case, the information entropy would not  be equal to the thermodynamic entropy, but the latter would be unchanged. (A better example would be if we happened to know something about the initial state -- for example, that all the particles in a gas in a box were originally in one side of the box. Then, at least in principle, we could evolve the probability distribution in time to find the final distribution, which would not be the same as the canonical ensemble. Nevertheless, even if we could do this (very hard) we generally do not choose to make use of this information, so for making experimental predictions we can still treat it as "information that isn't accessible to us").
Note: there is still a some amount of freedom in the definition of thermodynamic entropy, associated with what we mean by "information accessible to us". It's probably not right to call this "subjectivity", but Jaynes [1] did come up with some fascinating thought experiments where the definition of entropy depends on whether or not we are able to access the internal degrees of freedom of some hypothetical kind of atom. If we have the ability to access these internal degrees of freedom, but choose not to, then there are several definitions of entropy we are allowed to use. It's interesting to think about how this is consistent with statements like

if I assume a reversible transformation I would have $Q=k_b T\Delta S$ : the heat received by the system if he changes its entropy of $\Delta S$.

The answer is that "reversible" is defined to mean a process in which the total entropy of the universe is conserved -- hence which processes we call reversible actually depends on the definition of the entropy.
[1] http://www.damtp.cam.ac.uk/user/tong/statphys/jaynes.pdf
For more reading on this viewpoint on the thermodynamic entropy, see https://aapt.scitation.org/doi/10.1119/1.1971557
("thermodynamic entropy" is referred to as "experimental entropy" in that paper).
