Is there a distinguished reference system, after all? The equivalence principle, being the main postulate upon which the general relativity theory rests, basically states that all reference systems are equivalent, because pseudo forces can (locally) be interpreted as gravitational fields and it is therefore impossible for the local experimenter to decide whether he is moving, or being accelerated, or motionless. In other words: there is no distinguished, "motionless" reference system.
Question: doesn't the rotating water bucket (parabolic water surface) give us an indication of our rotational state? It would be a weird gravitational field indeed that causes my water to be pulled outward while causing the rest of the universe to rotate around me?
And doesn't the red/blue-shift of the microwave background (often dubbed "echo of the big bang") give us a clue of our translational motion within the universe (I read lately that they compensate the precision measurements of the background radiation by the motion of the solar system around the galactic center, obviously assuming that galactic center is "motionless" within the universe)?
 A: I'd like to answer as far as buckets are concerned but leave the CMBR to a cosmologist or a real relativity-ist. Mopping the floor up after the chaos left by my children, I think of myself as an expert on the former!
In GR it is immaterial whether one describes a "force" as an "inertial force" or a gravitational field. All one "knows" is whether one is accelerated relative to an inertial frame: more fully: suppose one carries a "reference frame" with oneself (imagine a set of straight measuring rods representing the $x, y, z$ axes. Then this system of rods is an "inertial frame" iff all of its points move along spacetime geodesics as defined by the Einstein field equations. This can be worded more technically: the co-ordinate's sytem's origin follows a spacetime geodesic in the manifold and the spacetime manifold tangent vectors represented by each rod are Lie dragged by the system of geodesics. 
A point on your bucket's rotational axis may well move along a geodesic, but there is, in GR, an "absolute" notion of rotation insofar that one could detect rotation with respect to the system of rods I described above. So all points on the rotation axes follow geodesics, but molecules of water away from the axis follow helices relative to the geodesics swept out by each point on our measuring rods.
Think of GR as an "application note" to go with Newton's and Euler's first and second laws. GR tells us what the inertial frames are: i.e. the definition of when and where Newton's first law and its rotational analogue apply. Then we apply Newton's and Euler's second laws (locally) to deduce all motion relative to these inertial frames: acceleration is then what we measure with an accelerometer. Whether we sit stationary relative to the surface of the Earth or accelerate uniformly in deep space at $g$ metres per second has the same physical description from this standpoint. Note that spacetime has to be locally flat, i.e. Minkowskian over the scale of the description of the accelerated motion for this way of thinking to apply (as it will be if we consider small enough "chunks" or "stages" of the motion): we can think of this description as applying in the tangent space to the spacetime manifold. In general, we'd have to update our inertial frame and reapply this thinking repeatedly if we follow an accelerated motion over longer scales in spacetime: this is what I meant by "applying locally".
Weird though it may be, if two spaceships met in deep space and were spinning relative to one another, from a wholly kinematic standpoint, we couldn't say which were rotating and which were not. But GR does not agree with this: one spaceship can be still relative to the Lie-dragged co-ordinates and so those riding in it will not feel a force and be in freefall with their lunch and coffee floating around them! Acceleration in GR is, in the sense (IMO the only one that matters) of what a system of accelerometers will tell us, absolute. See Mark Eichenlaub's wonderful answer here for more. I think this is likely what Lionel's answer meant about deep rabbit holes and Mach's principle - but, hey, you should follow Alice, drink all the bottles you find and look it up. Mach's principle, as far as I understand, tells against Einstein's general relativity and recently is being found to gainsay the observed results of gravity probe B (see Wiki page) so it is now falsified where GR is not, but it's thoroughly interesting from an historical perspective and from the standpoint of getting into the head of a very bright and altogether originally-thinking person (Ernst Mach).
Interestingly, the Lie dragging condition this is another way of saying that the torsion is nought in GR: other gravitational theories (which I know nothing of) such as Einstein-Cartan theory have nonzero torsion in some conditions although I believe in deep "empty" space there is still no torsion.
A: First of all your statement "because pseudo forces can (locally) be interpreted as gravitational fields and it is therefore impossible for the local experimenter to decide whether he is moving, or being accelerated, or motionless." is incorrect. 
I will paraphrase MTW's 'Gravitation', section 13.6, page 327:
We have a very small man inside a very small, sealed cabin, with apparatus bolted to the floor and to the walls. There is an x,y,z grid marked on the walls and floor. His apparatus consists of clocks, accelerometers and gyroscopes (like your buckets). He himself is not bolted to the walls or to the floor. Confined to his cabin, he cannot tell whether space is curved or flat or if any non-gravitational force field is acting on him. 
Now take the tapestry of space-time endowed with a metric (it must have a metric). If the metric is totally known, it is known for all of space and time. At every event, let us place small men in small cabins like above. Let us also allow multiple cabins to inhabit the same space-time points (remember these are very tiny cabins). 
Now given a man and a cabin at a point, there are 3 possible situtations:


*

*If the accelerometers give a finite value, his cabin is accelerating with respect to the non-inertial Lorentz frames of reference around his event and he will no longer float in the cabin.

*If the gyroscopes on the wall are moving with respect to the wall, then his frame is rotating with respect to the inertial Lorentz frames about his event and he will find himself no longer floating in the cabin and puking instead.

*If he is neither puking nor stuck the walls or ground, his frame is inertial with respect to other inertial Lorentz frames about his event.


All the cabins in (3) constitute the set of all motionless frames of the given space-time.
Only non-gravitational effects can yield the scenarios (1) and (2), because without them, all cabins would be free falling, non-rotating (allowing the little men to float and to not puke). Non-gravitational effects cancelling out at the event also cause (3) to happen (but that is identical to the absence of non-gravitational effects).
A: All the previous answers are correct. Let me add just some mathematics. Take a look at the equation for Fermi normal coordinates, for example in the original article by Misner and Manasse [1] or here. These coordinates provide an example of a local Lorentz frame, that is, a reference frame with a locally flat metric. As you can find in these articles, Fermi coordinates are valid provided the following "locality" conditions:
$$r\ll r_0\equiv \min\left\{{1\over |\boldsymbol{G}|},{1\over |\boldsymbol{\omega}|},{1\over |\overset{0}{R}_{\mu\nu\rho\sigma}|^{1/2}},{|\overset{0}{R}_{\mu\nu\rho\sigma}|\over |\partial_i\overset{0}{R}_{\mu\nu\rho\sigma}|}\right\}. $$
The first condition states that the local coordinates are valid for spatial distances $r=|\delta_{ij}x^ix^j|^{1/2}$ smaller then the typical length scales of the non-gravitational four-acceleration $G^\mu$ of the non-inertial observer. Similarly, the second refers to the typical scales of the four-rotation $\omega^\mu$ as measured by gyroscopes. The last two conditions determine the size of the region where the space can be considered flat despite a rotating or an accelerating observer by means of non-gravitational forces.
The rotating water bucket and the measurements of the CMB are two non local experiments, according to the previous constraints. In the former the radius of the circumference of the rotating bucket ($|\boldsymbol{\omega}|\sim 1$ meter) is equal to the length scale of your experiment (except if you're an ant on the bucket!). In the latter the experiment is sensitive to the curvature of the spacetime toward the centre of the Milky Way, whose length is shorter then that of the Universe.
A: GR treats all free-fall reference frames as equivalent. Also, a reference frame is a local thing, as you said. A rotating bucket filled with water is a non-local thing.
Anyway, you can go down a very deep rabbit hole by looking up Mach's principle, but I'm not sure I would advise this (I think it may be antiquated).
Finally, assuming the universe is infinite, or at least, is isotropic and relatively homogeneous in on a sufficiently large scale, there need not be a center of the universe. Present observations seem to suggest that every point in space is accelerating away from every other point at the same rate. So, ignoring local fluctuations, we could say that at some scale things are locally "at rest" in the same way that points on an the surface of an inflating cylinder can be "at rest". But suppose we set the cylinder spinning. On the 2-dimensional surface of this cylinder, there would be no way of detecting this motion. We can only measure our motion relative to other points on the cylinder.
Similarly, we can only measure our motion relative to distant stars/matter, but, as far as GR is concerned, we can't measure our motion relative to space itself. We can imagine a reference frame in which the entire matter content of the universe is moving 1 m/s to the left. If physics preferred a reference frame, we could detect this by, for example, noting that light travels faster in one direction than in another (as in aether-type theories). I think this is what is meant by preferred reference frames.
