Demonstrating that water density is at its maximum at 4 degree I want to demonstrate to the students this fact in class. I checked the density of water as a function of temperature. The variation is minute, on the scale of one in ten thousand, from 0 degree to 4 degree. 
This might mean this is a bit challenging. 
So, does anyone have a good suggestion or any experience? 
 A: I am not sure the following will work, but you may use the fact that water compressibility is very low, so a small water density change due to a temperature change in a fixed volume may cause a significant pressure change, which can be measured. 
A: You could use a setup similar to a Galileo thermometer, i.e., have a body with an (average) density which is below the density of water at $4^\circ\mathrm{C}$ but above that at e.g. $0^\circ\mathrm{C}$ and $8^\circ\mathrm{C}$: It would swim at $4^\circ\mathrm{C}$ and sink at both higher and lower temperatures.  This actually works, as witnessed by the Galileo thermometer; if you use a design for your floating body similar to the Galilean thermometer, it should be possible to tune the average density quite precisely.
It seems that the density difference between $0^\circ\mathrm{C}$ and $4^\circ\mathrm{C}$ is about $1/3$ of that between $20^\circ\mathrm{C}$ and $21^\circ\mathrm{C}$, so it should be possible to resolve this difference.  (Source.)
Of course, the same caveat as in my comment above holds: You need to assume that the thermal expansion coefficient of whatever body use use is negligible as compared to that of water.
