Can anyone explain to me what is actually "curved" when we speak of a Berry Curvature?
It's the curvature of a connection on a principal U(1) bundle over parameter space.
In describing the quantum Hall effect, we have a Hamiltonian, which depends on a number of parameters $H(R_1,R_2,..R_N)$. Suppose we have the system in its ground state. We now vary the parameters adiabatically (slowly!). As we vary the parameters, we can think of tracing out a curve $(R_1(\lambda), (R_2(\lambda)...(R_N(\lambda))$ in parameter space. As we twiddle the parameters we actually evolve the state using the Schroedinger equation. If we transport it round a closed curve in parameter space, i.e. we return to our starting parameters, we find that the state picks up a phase factor relative to the starting state. (Phase factors live in $U(1)$ the group of unit modulus complex numbers).
The mechanism that allows us to go from a state at one set of parameters to a state at another set is called a connection. In this case the connection is provided by the Schroedinger equation. Saying that the connection has curvature just means that transport of a state round a closed curve using this connection doesn't quite get you back to the state you started with (in fact mathematically the object which defines the curvature is obtained by transport round a little closed parallelogram).