Indeed, the age is not just $1/H$. (In this answer I'll use units with $c=1$.
The age $T$ can be calculated by making repeated use of the chain rule, as well as $da/dt=aH$ and $a=1/(1+z)$
\begin{equation}
T=\int_0^T dt = \int_0^1 da \frac{dt}{da} = -\int_0^\infty dz \frac{da}{dz} \frac{a}{H} = \int_0^\infty dz \frac{1}{(1+z)^3 H(z)}
\end{equation}
where I've assumed a flat universe so I can normalize the scale factor by $a(today)=1$, and I set the origin of time at the Big Bang a(t=0)=0.
However, using the Friedman equation and assuming a universe with matter, radiation, and a cosmological constant we can write
\begin{equation}
T=\frac{1}{H_0} \int_0^\infty dz \frac{1}{(1+z)^3 \sqrt{\Omega_{R,0}(1+z)^4+\Omega_{M,0} (1+z)^3 + \Omega_\Lambda}}
\end{equation}
Ignoring inflation, and using the $\Lambda$CDM best fit values for $\Omega_{I,0}$ ($I=R,M,\Lambda$), the integral is $O(1)$ (as you can check), so $1/H_0$ is a good approximation to the age of the universe.