Your logic is very good, indeed. And it makes sense, what you describe. I agree.
But then we try to measure it. And here comes the big problem: what we measure in our world doesn't follow this logic!
It seems very logic, but the world just doesn't behave that way. Weird, yes. But apparently it is true.
A famous example:
Put a light measurement device at the front and back of a car. Shine light on the car from behind. We can measure when it reaches the first and second device so we know the time it took light to move from one to the other. The distance between them divided by this duration gives the speed:
$$c=3\times 10^8 m/s$$
This is fast, but that's OK. Now, the car is driving. 300 km/h. We shine light from a lamp on the ground behind it. We measure the time from the light hits the first to the second device. We calculate again and get... the same answer as before:
$$c=3\times 10^8 m/s$$
We would expect to measure a smaller speed, since we would expect that it takes longer for light to pass over the car, now that it is moving. But we don't see that. In motion or not, the speed is the same. We would have thought that we could "outrun light" - so if we ran with half the speed of light, it would take longer time for light to reach us. But that is not what we measure! Light has the same speed and takes the same time to move the distance, nomatter how fast this distance tries to move away.
Many experiments like this show that the speed is the same nomatter how we measure it! And in other words, no matter from whose point of view!
Speed of light is constant, but not time
And now we could ask ourselves, what does that mean?
Consider a light bulb on the floor in a train cabin and a measuring device on the ceiling. The guy who sits in the train cabin sees the light shine from floor to ceiling. This is the total distance light covers.
A man on the station on the other hand, he sees the light leave the bulb, but before it reaches the ceiling, the train has moved to the side. So when the light hits the ceiling, it has moved not vertical, but at an angle. And this distance is longer than the vertical distance.
So, light covers two different distances in the exact same experiment. It depends on who you ask. But, for both of the people the speed of light is still the same $c$ (as the previous experiment showed). Since speed is the same, and distance traveled is different, the time must also be different.
So here we reach the relativity: Time is not the same for all - time is slower (you age slower) if you are moving faster.
It is something we measure. And it has been measured. So it is a fact of this world. But weird, very weird, and not easy to either explain nor accept for our brains. Luckily this relatively is only an issue at very, very high speeds. So on no normal life situation will this be an issue.