In diagrams which illustrate special relativity, can an observer "know" that he is moving? Or is it always assumed that an observer in an inertial frame of reference thinks that he or she is at rest, and that it is only others - observers and frames of reference - that are moving?  In examples which depict a moving ball and a traveling photon in a car of a train, which is itself moving at constant velocity along a train track, can an observer in the train car know that he or she is moving?
 A: If observer A thinks he is at rest and observer B is moving, he is right. Observer B, who thinks he is at rest and A is moving, is also right. 
You can choose any inertial frame of reference. When you do, motion as viewed by that frame is correct. 
This extends past velocity. Observer A says he is not moving. He is in a particular place at time t1 and the same place at t2. Observer B has a different view. A is moving. He is in two different places at those two times. Both A and B are right. 

Update - Some more explanation is in order. 
Suppose space was completely empty. You could not tell one point from another. They are all completely alike. Likewise, you could not tell one direction from another. All times would be alike. 

Suppose you (observer A) were the only object in space. Now you can tell one point from all the others. That point is the one where you are. And that is the only way to tell points apart. 
You can tell that you are not rotating if there are no centrifugal forces pulling one part of you away from another part. 
If you have rods, you can set up a coordinate system. You can now set up the rods to identify any point and give it a name based on how far it is from you. You can call the point where you are at a given time the origin or $(0,0,0)$. 
As long as you and your rods are rigid, there is very little point to talking about time. The universe is the same at all times. 
But you cannot say if you are moving or not. All points are alike. You cannot say whether or not the origin is at the same point in space at two different times. At any given time, you can say the distance between the points named $(0,0,0)$ and $(1,0,0)$ is $1$. 

Things change if the universe contains observers A and B. Both A and B can set up their own coordinate systems and give their own names to points. Neither A nor B know if their origins occupy the same point in space at two times. But they can measure the distance between the two origins, and see that it changes. 
You can say that B's origin is at a point named $(x_1, y_1, z_1)$ at time $t_1$, and $(x_2, y_2, z_2)$ at time $t_2$. You can say that B is moving with respect to you. That is different from saying B is moving. 
B is completely free to you his own coordinate system - his own names for points - and conclude that you are moving with respect to him. 
A: He cannot know, but he can ascribe himself either a state of proper rest or state of proper motion. The reflections below are to demonstrate, that two observers cannot ascribe themselves the state of rest simultaneously and that observations by moving observes are different from that who is at rest.
If two observers move relatively to each other with relative velocity v = 0.9 c ,in no way they can be at rest simultaneously, since they move relatively to each other. Whatever each of them thinks.
Relativistic observers A and B are equivalent in that sense, that each of them 
can think that he is „at rest“ and conduct observations from his rest frame, filled by Einstein – synchronized clocks. But A is at rest, B should not change his opinion of his state of motion, and consider himself being in motion.
Thus, if observer A ascribes himself state of proper rest, he adds another clock A2 at certain distance from his own A1, synchronizes clocks by light, admitting that velocity of light is c (Einstein synchronization convention).
In this rest frame velocity of light is considered to be c in all directions. 
Then observer A measures dilation of observer's B clock. Clock B1 passes by clock A1 first and clock A2 then. Two spatially separated clocks A1 and A2 will measure longer period of time than clock B1. If clocks B1 and A1 showed 12 PM at the meeting both, A2 will show 6 PM and B1 3 PM when their meet.
But, let‘s  observer B ascribes himself state of proper motion. What he does? He does not introduce his own rest frame (i.e. does not place another clock at certain distance from his own) but uses A‘s rest frame. He notices, that according to his clock, time in reference frame A runs gamma times faster than his own.
Reciprocity of observations and those miraculous effect like “You shorter than I and I shorter than you“ appear as soon as every relativistic observer covers space with his own rest frame with synchronizes clocks by the same method (Einstein“s synchronization). If they choose one frame and stay within that arbitrary chosen frame, moving observer would measure that time in reference frame runs faster and measuring rod „at rest“ stretches.
Now imagine that observers A and B play some kind of ping pong with photon. They agree, that they will launch a photon transversely to direction of motion.
Each of them possesses a tube. This tube combines a laser pointer and telescope.
Observer B ascribes himself state of proper rest and stays in the origin and launches a photon straight up along y axis.
Observer A moves at parallel line to axis x.  Photon had to be launched a bit prior than observer A crosses point Y of y axis, because it takes the same time for photon and observer A to meet at point Y.
Can observer A ascribe himself state of proper rest too? No, he can‘t. Why? Because he will not see the photon then.
Observer A MUST tilt his telescope at oblique angle $\theta_A$ if he wants to see the light. It is like Bradley tilted his telescope a bit into front and observed, that distant stars change their position. It is because of aberration of light.  This angle depends on relative velocity of observer, and source of light appears in the front of him. Angle of emission and angle of reception are tied with relativistic aberration formula.
$$ \cos {\theta_A} = \frac {\cos {\theta_B} - \frac v c} {1- \frac v c \cos \theta_B} $$
Let‘s consider, that observer A rotates around observer B. Observer B emits light. In this case observer A always has to keep his telescope at oblique angle, because flux of radiation comes at oblique angle to A. Photons will always be blueshifted at $\gamma$. 
If A emits light, the light will always come at right angle to B. Photons will always be redshifted at $\gamma$.
This article in Wikipedia (chapter Transverse Doppler Effect) explains, that moving observer (light received at points of closest approach) will see blueshift (clock runs faster than your own) of radiation, and observer „at rest“ (light emitted at points of closest approach) will see redshift (clock runs slower than your own) respectively.
That is fully in accordance with dilation of that clock, which is "in motion" in chosen reference frame.
https://en.wikipedia.org/wiki/Relativistic_Doppler_effect
You may ask, what will happen if the both observers will chose a reference frame, in which the both move with equal velocities. Well, they will not see any time dilation, because A has to tilt his laser pointer backward and B has to tilt his telescope forward at equal angles.That simply means, that their clock dilate at the same magnitude since they move with equal velocities relatively to one mutual REFERENCE FRAME.
http://mathpages.com/home/kmath587/kmath587.htm
"A simple way of expressing this is to point out that the null frequency shift occurs for the pulse that travels the shortest distance from emitter to receiver, and this pulse is obviously neither emitted nor received at the point when the emitter and receiver are at their point of closest approach. Since the pulse requires an amount of time to traverse the distance between emitter and receiver, it must be emitted slightly before the point of closest approach and received slightly after. Of course, the spatial distance traveled by this pulse (for emitter and receiver on parallel paths) is equal to the minimum distance between the emitter and receiver, i.e., the perpendicular distance between the paths"
Animation:
https://www.youtube.com/watch?v=hnphFr2Iai4
https://www.youtube.com/watch?v=AGMINcBYojc
Observers A and B can choose a frame, in which A is at rest.  A keeps tube straight up, B keeps tube „into front“. A measures dilation of B‘s clock, B measures that A‘s clock run faster.
Observers can choose a frame, in which B is at rest. B keeps tube straight up, A keeps tube „into front“. B measures that A‘s clock run slower, A measures that B‘s clock runs faster.
Observers can choose a frame, in which A and B move with equal velocities. A keeps tube backward, B keeps tube into front. They measure no dilation
They cannot keep their tubes „up“ and „up“ respectively. They cannot be at rest together, since they relatively move. Frame in which they both are at rest does not exist.
