To understand what field is "real", write a charge equation of motion. The force in it is determined with the real field there. In a medium it is **E**: $ma = eE$. In case of magnetic field, it is $B$ that determines the force.