This is Dirac's formalism. It is a generalization to continuous basis, i.e., those enumerated by an index which takes values in a continuous set like $\mathbb{R}^n$.
In that setting you have that in the same way that $|e_n\rangle$ is a discrete basis which allows you to decompose
$$|\psi\rangle=\sum_{n}\langle e_n|\psi\rangle |e_n\rangle,$$
and with respect to which the closure relation holds
$$\sum_{n}|e_n\rangle\langle e_n|=\mathbf{1}$$
we suppose that we can have one basis $|x\rangle$ enumerated by some continuous parameter like $x\in \mathbb{R}$, such that we can decompose
$$|\psi\rangle=\int \langle x|\psi\rangle |x\rangle dx$$
with closure relation
$$\int|x\rangle \langle x|dx = \mathbf{1}.$$
The issue is that roughly if $A$ is one compact operator, the spectral theorem will ensure you have a discrete orthonormal basis of eigenvectors $|a_n\rangle$ such that $A|a_n\rangle = a_n |a_n\rangle$ (I assume non-degenerate for simplicity).
When $A$ is unbounded as often happens in QM, no such basis exists. But you suppose that the above sort of generalized basis do exist. So if $X$ is unbounded you assume that for every eigenvalue $x\in \sigma(X)$ the spectrum of $X$ there is a state ket $|x\rangle$ with $X|x\rangle = x|x\rangle$ and forming a basis.
Notice that whenever you have position and momentum operators $X,P$ you want to require $[X,P]=i\hbar$ and there is a theorem which ensures that at least one of them will be unbounded, so the thing above will be needed then.
This is important due to the postulates of QM. Observables are hermitian operators. The possible values to be measured are exactly the values on the spectrum, i.e., the "eigenvalues" and the states with definite value of the quantity are the eigenvectors, the value measured is then the corresponding eigenvalue.
It is then postulated that if $A$ is the observable with continuous basis $|a\rangle$ then $\rho(a) = |\langle a|\psi\rangle|^2$ is the probability density of finding the value of $A$ in the state $|\psi\rangle$ to be between $a$ and $a+da$.
You then connects this with wave mechanics. Consider a particle in one dimension. We have the observable $X$ corresponding to position. Let $|S(t)\rangle$ be the state at time $t$. As we know position can assume any possible value so $\sigma(X)=\mathbb{R}$. Let $x\in \mathbb{R}$, the corresponding generalized eigenvector is $|x\rangle$. The probability of finding the particle between $x$ and $x+dx$ is then $\rho(x) = |\langle x|\psi\rangle|^2$.
So making contact with wave mechanics, we see that $\Psi(x,t)=\langle x|S(t)\rangle$ indeed.
As a final remark, all the thing about generalized eigenvectors and continuous basis from Dirac's formalism is extremely useful and elegant, but it is non rigorous. In rigorous functional analysis there is no eigenvector for unboudned operators and these expansions are not defined. There is, though, one workaround that makes it all make sense, called the Gel'fand triplet approach.