Well relativity teaches us that time interval between two events is a frame dependent quantity, then how can we say that our universe is 13.8 billion years old? Should it not depend on who is asking this question?
Let me start by pointing out that you are mixing a principle from a local theory (special relativity) with a conclusion from a global theory (Big Bang cosmology) and that is where the confusion stems.
That said, special relativity does have its place in cosmology as a whole, so we can use it to our advantage here. Let's examine a simple scenario to begin with from a special relativity stance: two stars supernova and two observers watch. Say the two stars are 10ly apart. Observer Alice is at a point 10ly from each star so the three form an equilateral triangle, while observer Bob is at a point colinear with the two stars, 10ly away from the nearest one.
The two stars explode, and one night Alice looks up at night and sees one supernova, then ten years later sees the other. Bob, on the other hand, looks up one night and sees two supernovae at once. He records a vastly different time interval than Alice does. However, Bob knows the distance to one star is greater than to the other and that light travels at a finite speed, so he deduces that even though he saw the two events simultaneously, an observer at infinity would see one before the other. I.e. he knows that one supernova is 'younger' than the other, and measures this by distance. This is a general consequence of a finite speed of light: we can map out the four-dimensional information and say that in a comoving frame, the events of the two supernovae occurred at the same absolute $t$ coordinate, and the events of the observations of the two supernovae occur at different $t$ for different observers, as a function of their distances from the supernovae.
Now, other effects complicate things a bit - for example, a third observer colinear with the stars and Bob but moving along that axis would record a different interval between the two supernovae than Bob would - but in any case we can Lorentz transform from any one inertial frame to another without problem, and even transform into accelerated frames if we're clever with the math. What it comes to in the end is that once we know the positions and velocities of the bodies involved, we can stop viewing a system of events as a dynamic system that changes over time to a static system that includes time coordinate as a fundament.
Now, let's apply this to cosmology. We can make observations of other galaxies and note that the appear to be moving away from us, and that the farther away they are (measured by deductive methods such as apparent brightness of standard candles) the faster they are moving away from us. This appears to us as though the universe is expanding and that we are in the center. However, with our knowledge of the distances between us and various bodies, and using basic trigonometry, we can transform into the frame of one of those other galaxies and from that vantage point, nothing really changes. That is to say, an astronomer in a distant galaxy also sees the universe expanding with them at them center. So whatever the universe looks like to us, that's how it looks to everybody.
Now, remember how it worked out earlier that when Bob looked at the supernovae, the he knew the nearer one was younger because it was closer, while seeing both at the same time? Apply this to the entire sky: we can see objects billions of light years away. We know light travels at a finite speed, so we also know that by looking far away we're also looking back in time, in a way. We're seeing it not as it is, but as it was. Likewise, a distant astronomer looking towards Earth would not see me typing this answer; they would see the Earth as it was long ago. Far enough away and they'd see no Earth at all, only a planetary nebula around a young star. Farther still and they'd see only the hydrogen cloud, or the supernova that cast off the cloud, and so on.
But in every direction, we can only see so far: to the cosmic microwave background. And if the currently popular cosmology is true, then somewhere behind that opaque background is the beginning of the universe. And you can't see farther back in time than the beginning no matter which observer you are.
Yes, you're quite correct it does depend on who is asking the question. When we say the age of the universe is 13.8 billion years we mean that a comoving observer will measure the comoving time from the Big Bang to be 13.8 billion years.
Have a look at my answer to Doesn't dating the universe violate the concept of spacetime's inseparability? as this discusses how we define time in modelling the universe. When we write down an expression for the geometry of the universe we normally use a coordinate system called comoving coordinates, and a comoving observer is one whose position (in space) is constant when expressed using the comoving coordinates. In real terms this means (roughly) that the observer is stationary with respect to the cosmic microwave background.
All comoving observers share the same time, that is once synchronised their clocks will all agree what the time is. The age of the universe is expressed in this comoving time. If you had spent the time since the Big Bang razzing around in a spaceship at high velocity then you would measure a small time and you would conclude the universe was younger than 13.8 billion years.