Throwing a ball in a rotating space station I have lots of trouble understanding circicular motion, fictitious forces, and how to dermine if a reference frame is intertial. I have thought of the following the scenario which causes some confusion. Suppose you are standing in a cylindrical space ship which rotates to simulate gravity (an O'neill cylinder for example). If you throw up a ball directly above your head, what happens? 
Once the ball leaves your hand, there are no forces acting on it (assume air resistance is negligible). Naively I would think that the ball continues to travel at constant velocity, but this conclusion is likely wrong.
To extend the scenario, imagine that we placed the ball in space, and constructed the space ship around the ball, then begin rotating the space ship. Everyone standing in the spaceship would feel 1g exerted on their feet, and so the gravity simulation would seem to be working well. But then they see the ball floating above their heads, with no forces acting on it, and they would wonder why the simulation does not apply to the ball.
 A: Suppose the ball is shot "upward" (i.e. a force is exerted in the radial direction) from a point on the floor. From the perspective of an external inertial observer, it will indeed travel in a straight line, in the direction of the velocity vector at the time it left the floor, a combination of the tangential velocity from the rotation and the radial velocity from the force that shot it "up". This means its path will be a chord of the circle defined by the station's floor, so from the perspective of someone on the floor it will appear to rise to some maximum height, then "fall" back down towards the floor. This page has a good illustration:

As noted on the page, the ball has a greater total speed and a shorter distance to travel than the point on the floor it was shot up from, so it won't land back at exactly the same point on the floor, an effect that in the station's rotating frame would be explained in terms of the Coriolis force, a velocity-dependent "fictitious force" seen in rotating frames, along with the centrifugal force.
This page has a similar diagram for a ball dropped from some fixed height above the floor:

In the extended scenario where the ball is at rest relative to the station's center (which could be achieved just by calculating the correct force to exert on it to bring it to rest in this frame), it won't hover above a constant position on the floor, since the inertial observer sees the floor rotating past it. So from on board the station, it appears to be flying sideways relative to the ground, maintaining a constant height, as if in an "orbit". Again, its failure to fall downwards can be explained in the rotating frame in terms of the fictitious Coriolis force.
For the actual equations, first define $\omega$ to be the angular velocity of the station's rotation, as seen in the inertial frame where the center of the station is at rest (if $T$ is the time period for one full rotation, then $\omega = 2\pi / T$). Then in the rotating frame where every point on the surface of the station is at rest, the centrifugal force on an object of mass $m$ at a radius $r$ from the center will have a magnitude given by $mr\omega^2$, and the the centrifugal force vector's direction is always pointing radially outward from the center. The Coriolis force on an object with mass $m$ and velocity $\vec{v}$ in the rotating coordinate system is given by $-2m (\vec{\Omega} \times \vec{v})$, where $\vec{\Omega}$ is a rotation vector whose magnitude is equal to $\omega$ and whose direction is parallel to the station's rotation axis (which end of the cylinder this vector points towards is determined by the right-hand rule, see the diagram here). And if you're not familiar with vector operations, the $\times$ symbol in $(\vec{\Omega} \times \vec{v})$ is meant to denote the cross product of the two vectors, which gives a new vector with magnitude $\omega v \sin (\theta)$ (where $\omega$ is magnitude of $\vec{\Omega}$ i.e. the angular velocity, $v$ is the magnitude of $\vec{v}$ i.e. the speed, and $\theta$ is the angle between the two vectors $\vec{\Omega}$ and $\vec{v}$), and direction given by the right-hand rule (see this diagram from the wiki page for the right-hand rule, or the diagram here with a different hand orientation that gives the same result). Note that because of the negative sign in the Coriolis force equation, the Coriolis force points in the direction opposite to $(\vec{\Omega} \times \vec{v})$.
If you work this out, you see that if an object has a velocity vector that's tangent to a circle around the rotation axis (like a circular cross-section of the station's inner surface), and in the opposite direction to the station's rotation (so if the station is rotating clockwise, the object's velocity vector should point in a counter-clockwise direction), then the coriolis force vector will point radially inwards, opposite to the centrifugal force vector which points radially outwards. The magnitude of the Coriolis force in this case will be $2m\omega v$, and since the magnitude of the centrifugal force vector is $mr\omega^2$, this means that if $v = r\omega$ the Coriolis force will be $2mr\omega^2$, twice the centrifugal, so their sum will be a force $mr\omega^2$ pointing towards the center. And as it happens, the condition for a circular orbit is that if the orbiting object has a radius $r$ and a purely tangential velocity $v$, it should experience an inward force of $mv^2 /r$; in this example, since the object has tangential velocity $v = r\omega$ the required inward force is $m (r\omega)^2 /r = mr\omega^2$, which is exactly what we found for the sum of the centrifugal and Coriolis forces. Therefore, an object with this velocity in the rotating frame will be predicted to orbit at constant radius, never "falling" any closer to the floor of the station. And the period for this orbit will be $T = 2\pi r / v = 2\pi r / (r\omega) = 2\pi / \omega$, which is exactly the same as the period of the station's rotation as seen in the station's inertial rest frame. So this means that the object which is "orbiting" in this way as seen in the rotating frame is simply at rest in the station's inertial rest frame, just as in your extended scenario.
A: General remarks.
Essentially all of your confusions seem to stem from erroneous mixing of observations in different frames of reference.  In particular, you must be careful not to conflate the observations of an observer rotating on the space ship with the observations of an inertial observer not rotating with the space ship.
To an inertial observer, the ball will indeed move along a straight line at constant velocity once it has been let go, but to an observer rotating with the spaceship, the path of the ball will not generally look straight.
A simple example to test your intuition.
A specific, simple case of initial data that makes this crystal clear is as follows:  suppose that someone on the edge of the space ship rotating with the spaceship throws a ball tangent to the edge of the ship and in the plane perpendicular to its rotation.  Suppose additionally that it is thrown opposite the direction of rotation with a speed exactly equal to that of the edge of the ship as viewed from inside.  In this case, from the point of view of an inertial observer on the outside, the ball's initial velocity will be zero because the velocity of the ship and the velocity of the ball cancel at the initial moment.  However, notice that the ship will be rotating under the ball from the inertial observer's perspective. This implies that from the point of view of the thrower rotating with the ship, the ball will travel in a circle along the rim of the ship.  
In other words, a ball just sitting at a point on the rim as viewed by an inertial observer seems to travel along a circular arc according to a rotating observer.
Another way of thinking about this is that it is only the inertial observer who observes no forces on the ball.  An observer rotating with the ship will observe forces on the ball, despite the absence of physical interactions between the ball and other objects.  These forces are not real in that sense, but they certainly look and feel just as real as forces due to interactions if you are a person rotating with the ship viewing a ball that has been thrown.
Mathematical detail via a cool problem.
The mathematics of all of this present in the following problem:
http://www.phermi.com/space-station-catch
and its detailed solution
http://www.phermi.com/space-station-catch-solution
Check out the images at the end as well which compare the paths of a ball in both the inertial and non-inertial frames.
A: 
I have lots of trouble understanding circicular motion, fictitious forces, and how to dermine if a reference frame is intertial.

What is a reference frame? There are some formal definitions. But, let's pick an intuitive one to boost understanding: 


*

*Definition: An isolated particle is a particle where the distance of the nearest particle is infinitely away. Therefore, an isolated particle will experience no interaction with another particles, and therefore no force.

*Definition: An inertial frame of reference is a frame of reference in which an isolated particle is with a constant velocity $v$.




Once the ball leaves your hand, there are no forces acting on it (assume air resistance is negligible). Naively I would think that the ball continues to travel at constant velocity, but this conclusion is likely wrong.

If you are in a reference frame, observing your space station, when someone throws a ball, Newton's second law says: $\mathbf F = \mathbf{\dot p}$. Which means, if there's no force on the ball, linear momentum $\mathbf p = m\mathbf v$ will conserve. Therefore, the ball will travel at a constant velocity, with respect to an inertial frame of reference. However, for those on the station, where their reference frame isn't inertial, they will see the ball curving in it's trajectory, appearing it has a force.


To extend the scenario, imagine that we placed the ball in space, and constructed the space ship around the ball, then begin rotating the space ship. Everyone standing in the spaceship would feel 1g exerted on their feet, and so the gravity simulation would seem to be working well. But then they see the ball floating above their heads, with no forces acting on it, and they would wonder why the simulation does not apply to the ball.

The ball in this case is an inertial reference frame. If you do this, the ball won't fall. Because, your analysis is correct, because you are a inertial frame of reference. The rotating people in the station in this case, is not. On their reference frame, they will see the ball rotating the station while they are not moving. Which means, the ball is accelerating in this case, while they are not moving. Since this force actually doesn't exist (because people are not a valid inertial reference frame), they are called inertial forces. If in a particular example the ball is exactly in the rotation axis of the station, they will see the ball not moving, while they not move. But, the ball also will not fall, because in their reference frame, the "gravity" depends on the radius $r$ from the center to the surface. And, since the ball is exactly in the axis, $r=0$ and no gravity for the ball in this non-inertial frame.
Just for the record, in the non-inertial reference frame of the people in the rotating space station, the non-inertial acceleration responsible for "gravity" is:
$$
g = \omega^2 r
$$
Where $\omega$ is the angular speed of the station, and $r$ the distance of a person from the central axis.

Let's explore the non-inertial frame of reference, of the people in the station, in those both scenarios. All non-inertial forces are the following:
$$
\mathbf g = \mathbf\omega\times (\mathbf\omega\times\mathbf r), \quad\quad
\mathbf a_c = 2\mathbf\omega\times\mathbf v
$$
Where $\mathbf g$ is the centrifugal gravity acceleration, and $\mathbf a_c$ is the coriolis acceleration, $v$ is the velocity of the ball in this non-inertial reference.  Those are vectors. The cross product $\times$ between the vectors returns a vector perpendicular to both simultaneously. Therefore, in a not moving ball with respect to inertial frame of reference, the total non-inertial accelerations are:
$$
a_t = 2\omega v - \omega^2 r
$$
The direction is radial, pointing to the central axis. Therefore, we can see that centrifugal gravity pushes you away, and Coriolis inwards. If ball is in central axis: $r = 0$ and $v = 0$, no acceleration, will be there. If ball is not in central axis: $a_t = 2\omega v - \omega^2 r$, since this case Coriolis will be bigger, will act like an "centripetal force" preventing it from reaching the ground, and keeping it's movement circular.
A: Grant Hutchison of the Cosmoquest Forums has some really helpful diagrams, though I unfortunately can't reproduce them here. He says:

Firstly, if you face to spinward, the trajectories of thrown objects are bent towards the floor of the habitat. Here's a selection of trajectories for horizontally launched projectiles:
  [Diagram]
If you face to antispinward, things are more interesting:
  [Diagram]
  Trajectories are lifted by Coriolis and start to curve around the habitat. A launch velocity equal in magnitude but opposite in direction to the habitat rotation speed will in principle send a projectile right around the habitat at the same height above the floor, endlessly - in practice, of course, it would slow and fall because of air resistance.
If you throw an object upwards and want it to return to your hand, you need to throw it a little to antispinwards, producing a range of trajectories with different launch velocities:
  [Diagram]
If you throw upwards and to antispinwards you can produce a range of interesting trajectories simply by varying the launch angle but keeping the launch velocity the same:
  [Diagram]
At first you retain the loop at the apex of the trajectory, but then you reach a critical combination of velocity and angle at which the loop disappears, giving the trajectory a sharp point at which the thrown object becomes momentarily stationary in the rotating frame.
If you want an object to strike the floor directly below the launch point, you need to throw it to spinward rather than drop it. The trajectories take wider loops the closer you are to the axis of the habitat: [Diagram]
If you want a vertically thrown object to land below its launch point, there is only one velocity that can achieve this for a given radial position within the habitat; too slow, and it will land to antispinward, too fast and it will land to spinward. Again, the trajectories become wider the closer to the centre of the habitat you launch from: [Diagram]

