The time used in describing the evolution of the universe is comoving time. This is the time that would be measured by a freely moving observer on their wristwatch (assuming the high temperatures didn't melt both the observer and the wristwatch :-).
Time is not a simple thing to define in general relativity, however we can always unambiguously define proper time. This is the time that would be measured by an observer who is floating freely i.e. not subject to any external forces. The proper time is an invariant and would have the same value for all observers.
The comoving coordinates are defined so that the time is the same as the proper time for these freely floating observers. When we say the universe is 13.8 billion years old (or whatever the current estimate is) we mean this is the time measured by these freely moving observers. You and I are approximately freely moving observers because the various peculiar motions due the the Earth moving round the Sun, Sun moving round the Milky Way etc are small compared to the speed of light. We are approximately at rest with respect to the cosmic microwave background.
Even though the expansion of the universe was extraordinarily rapid during inflation, any observers present would still have been freely moving and would still measure time in exactly the same way you and I measure time today. When we quote a figure for the duration of the inflationary phase it's this time that we are using.
Response to comments:
I think it's worth expanding on the discussion about the meaning of time in the comments. If we make a few simplifying assumptions about the early universe (principally that it was isotropic and homogenous) the early universe was described by the metric:
$$ ds^2 = -dt^2 + a^2(t) \left(dx^2 + dy^2 + dz^2 \right) $$
were $t$, $x$, $y$ and $z$ are comoving coordinates and $a(t)$ is the scale factor. It's important to emphasise that these coordinates are just one of the many choices we may have made so there is not necessarily any physical significance to them, though as you'll see they do have physical significance.
This is called the FLRW metric - I think strictly speaking the metric doesn't apply to the inflationary period, but all inflation did was change the scale factor $a(t)$ by a factor of $e^{50}$ or so in a short time. The form of the metric remains the same.
Note also that $t$ is just a coordinate like $x$ etc. It starts at zero and extends to infinity and is continuous in between so it is defined at all points in spacetime no matter how closely spaced they are.
Now, the definition of a comoving observer is that they don't move in space so $dx = 0$ etc. That means for a comoving observer the metric simplifies to:
$$ ds^2 = -dt^2 $$
The proper time is defined by $d\tau^2 = -ds^2$ so:
$$ d\tau = dt $$
And this immediately integrates to give:
$$ \tau = t + C $$
where $C$ is the constant of integration that we can set to zero by setting our watches appropriately.
The point of all this is that the proper time agrees with the elapsed time for a freely moving, i.e. comoving, observer. So the observer time is the same as the comoving time. This emerges from the metric and doesn't rely on the observer having some form of clock. The observer's time is just a coordinate like comoving time and takes all values from zero to (potentially) infinity.