Now I am wondering compared to what reference point is this speed
measured?
There is no special reference point. Basically you can pick any inertial reference frame as the reference and do the calculations using the velocities of the clocks you want to compare relative to your chosen reference frame and everyone will agree on the proper time that elapses on the clocks you are observing, between chosen events. I will elaborate on this below.
Is it absolute speed in comparison to the spacefabric (is that even
possible to measure?)?
There is no absolute speed. Only the speed you observe relative to your chosen reference frame. Rather than a spacefabric, we have spacetime, which is defined by a hypothetical or real grid of rulers and synchronised clocks that are all at rest in our chosen reference frame. Once a spacetime reference frame has been chosen, we can measure the path of a moving clock relative to the chosen reference frame. For example if the clock starts at (x1,t1) and ends at (x2,t2), where x is measured by rulers at rest in your chosen coordinate reference frame and t is measured by clocks at rest in your chosen reference frame, then its path length through spacetime is defined by $\sqrt{\Delta t^2 - \Delta x^2}$, where $\Delta x = (x2-x1)$ and $\Delta t = (t2-t1)$. This is sometimes known as the invariant or spacetime interval but it represents the proper time that actually elapses on the moving clock. The whole experiment can be repeated in another reference frame and they will get the same result.
I'd imagine it to be similar to me running back and forth on a plane.
In the end my average speed will be the same (If I sit back down on my
place) compared to me sitting down the whole time. Does that mean,
that time dilation was the same?
In this case we could choose the reference frame of a passenger that remains seated on your aircraft. On his spacetime chart, he is stationary and his path is straight up because he is only advancing in time, while your path would be zigzag path on his chart, due to running back and forth relative to him. Now when you add up all the segments of your zigzag path using the spacetime interval formula it turns out your path (and elapsed proper time) is shorter than his. Initially it might seem odd that a zigzag path is somehow shorter than a straight path but this is because the formula has a minus in the square root unlike the addition we normally use to calculate distances using Pythagoras. Now a person on the ground will see the seated passenger moving at a great velocity relative to him, but the seated passenger will still be following a straight (but tilted) path in his spacetime and you will still be following a zigzag (but tilted) path through his spacetime and he will also calculate your proper time to be less than that of the seated passenger.