Strictly speaking, interpreting the expression as having anything to do with probability of measurement of particle at a single point is not correct.
Quantum theory uses point particles, but their description in terms of psi function cannot ever appear to be localized to a single point of space (or a system of particles cannot be localized to a single configuration). The reason is, the way the Born probability interpretation works is that $|\psi|^2$ gives probability density that when integrated over all space, gives 1. But in mathematics there is no normalizable psi function that could describe particle localized to a single point in a consistent way; there is no "square root" of delta distribution.
More correct way to understand things such as $\langle x,t|x_0,t_0\rangle$ is the way they were introduced or defined in the first place: as Green's function $G(x,t;x_0,t_0)$ of the time-dependent Schroedinger's equation. It is a function that solves the time-dependent Schroedinger equation for $\psi(x',t')$ for $t'>t_0$ for initial condition $\psi(x',t_0) = \delta(x'-x_0)$.
This Green's function allows us to express future psi function that comes out as result of evolution of any given initial condition psi function $\psi_0(x)$, as an integral of this function:
$$
\psi(x,t) = \int G(x,t;x',t_0) \psi_0(x') dx'
$$