I would like to know how do people come to derive physical laws out of experiments alone, for example how did Newton came up with the laws of gravity? Did he just set those as axioms or did he observed them through experiments? If the second is true, then one way I could think he did is that he used statistical methods to collect data and out of those derived the physical law, but such a way could not provide one with accurate results. Was this the case? Can you explain how do scientists actually use experiments to derive physical laws?
It's a mixture of the two. A crude idealization of the scientific process is:
- Scientist finds unexplained phenomenon.
- Posits a theory that would explain the phenomenon, and makes additional testable predictions. The goal is not just to explain what we've already seen, but predict things we haven't tried yet.
- Start testing its predictions. If they're good, then we have a useful theory.
As far as Newton's law of gravity is concerned, the motivation for it came a combination of noting that free fall times did not depend on mass (which implies the $Mm$ part) and astronomical observations. The observation of Kepler's laws predated Newton, so it remained only to be shown that an inverse square law in gravity gives Kepler orbits, and that extended masses such as planets could reasonably be treated as point particles to posit the full law of gravity. There's some historical controversy over who exactly did what first, but at any rate Newton was the first to publish the theory in its full form.
At that point, it would hardly be considered a law the same way it would today. Newton's theory would not be directly tested experimentally (and $G$ measured) until much later, more than 70 years after Newton's death.
[edit: this answer was written when the title and content of the question were regarding the "general" connection between experiments and theories]
Well, it's a difficult question which would require a very long and accurate answer, so I will just try to throw some ideas.
First of all, "physical" (in the senso of "related to nature") laws have been around for centuries - except many where wrong.
It used to work somehow like this: I observe something and I have a given framework of ideas (a belief, a religion, etc.), I match the two things together and voila'. Anybody can one wake up one day a declare a new natural law. Of course, people will believe the law only if it does not openly contradicts some other observations (or their belief).
For example: -rocks fall down, bubbles go up. The way Aristotle explained it, was that air-bubbles have a (meta)physical tendency of going back to their "realm", i.e. air, so if you put them in water, they go up. Rocks also want to go back their "natural place" (i.e. Earth) so they fall down. It's a nice explanation and people believed it for a while, but then they started noticing contradictions (why does wood float? why do flames go up? etc.) and this theory was abandoned. -the sun goes up every morning and down every night. If you assume Earth is still and the sun goes down and you develop a (very convoluted) mathematical system, not only you explain it in a way which makes sense also on a metaphysical level (God created Earth -> Earth must be the center of the Universe) but you are also able to predict several observable quantities (e.g. when the sun will set or where will a given star be at a given time...).
So what do we learn here? -Theories come from observation. Sometimes they come from a mental framework (God is good->man has free will, for example) but then they have to, at least a little bit, be related to things we observe every day (e.g. that we seem to be free in our decisions).
-Experiments (or observation) can make a theory crumble when too much data has been accumulated that openly contradicts the theory.
So a way in which a theory can span from experiments is a) I make several experiments, I see a trend, something which happens all of the time and then I declare it a theory b)I have a theory, I do experiments which contradict it so I have to make a new theory to explain the new experiment.
Let's see what happened in Newton's case:
In order to arrive at Newton's time we first have to take a brief detour.
Aristotle (yet again) and his disciples had observed that if you push an object, the object moves, so he postulated that (as we would say today) Force is proportional to velocity ($F\propto v$). This now seems naive, as we know $F\propto a$, but if you think about it, Aristotle's law reflects most of everyday's experience, i.e. objects falling down, objects falling in water, pushing objects etc., because (we know now) there is friction and because of terminal velocity objects do seem to fall at constant speed (the transient time of acceleration is very small!)
Then, maybe to prove this theory, maybe because he noticed something was off, Galileo started making several experiments on an incline in which friction was constantly diminished, by polishing, and noticing that the velocity, at constant force, was far from constant, i.e. the object was accelerating. In doing this Galileo recorded a lot of data and recorded them in a mathematical formulation. This was indeed his great belief: that Nature could be described with mathematics and (though not really his) it was a successful idea: the experiment, when put in a mathematical representation, clearly indicated the $F=ma$ relationship.
So we now have to add another ingredient to the experiment-to-theory thing: math! What clearly defines physics is the idea that nature can be described in mathematical terms which allow to a)verify if an experiment fits a theory (and vice versa) b)make predictions starting from a theory c)quantify how much the prediction was precise.
I admit I don't know many details of how Newton came to his principles, but it surely was a mixture of observation (the famous fallen apple, though a myth, reflects that) and mathematical brilliance (to develop his theory he firstly had to invent calculus - though at the same time Leibnitz invented it too apparently indipendently). Using only Newton's principle (and his Law of gravity), most of classical mechanics can just be computed mathematically. How does a pendulum work? How do falling objects fall? let's just use Newton's principle, some diagrams, some maths, and we get the results. Does it really work? We can do an experiment and quantify.
As far as gravity is concerned, that idea spans from several experimental facts (indipendent free fall time for objects of different mass, Kepler's and Copernicus' theories, etc.) but it also required a big "mental" step, i.e. that of action at a distance (which led to the idea of "force field"). So experiments (observations) can lead to theories, if you add a little bit of ... inspiration.
So in this case experiments allow for the empirical verification of a theory. And Newton's theory was correct!
And then came Einstein. We like to think that Einstein was a theoretical genius, but many neglect the fact that he was very convinced of the importance of experiments. The need for Einstein's theory of relativity came from several experiments (amongst which Michelson's and Morley's) which were indicating that Maxwell's theory of eletro-magnetism was... wrong. And so was Newton's/Galileo's theory. Well, while wrong is the right word it does not do it justice: let's say that theories are approximation of the truth, which at a given point fail because we are able to discern details that were before undetectable.
This is way Newton beat Aristotles and Einstein beat Newton and so on.
And experiments are crucial in that they allow us to a)expand a theory b)verify a theory c)prove a theory wrong
So physics is a convoluted system: theories inspire experiments, experiments modify theories, and so on and so on, and everything also depend on the social, economical, religious etc. background of the people doing science!
Let me state, as a conclusion, that there is another part of experiments which is interesting, also in the context of a well established theory: they allow a scientist to concretely see that their view of the world is right (or, rather, precise enough), they inspire new ideas (maybe you something that you weren't expecting), they validate, they are beautiful!
This is it, in a nutshell, with little historical accuracy but a lot, I hope, of inspiration (:
Can you guess that I actually am an experimental physicist? :D