The quantum state of a quantum mechanical system can be described with a vector $|\psi\rangle$ of a Hilbertspace $\mathcal H$. If you want to characterize the quantum state completely, you have to introduce a "complete set of compatible variables".
What does that mean ?
In Quantum mechanics every physical variable is represented by an hermtian operator, called observable. Physical variables in classical mechanics are functions of position and momentum, e.g.
$$
A(\mathbf{p},\mathbf{x}).
$$
In quantum mechanics you have to replace $(\mathbf{p},\mathbf{x})$ with the momentum and position operator $(\hat{\mathbf{p}},\hat{\mathbf{x}})$. But there are quantum mechanical variables, like the spin, which does not have a so called "classical analogue". Measurable values of an observable are its eigenvalues. If the quantum state $|\psi\rangle$ is an eigenvector of an observable, e.g. the position operator $\hat{\mathbf{x}}$ the location can be measured as precisely as required. E.g. the quantm state of an electron in a hydrogen atom is characterized by three quantum numbers; n,l,m. These numbers are related to the eigenvalues of the Hamiltonian (Energy) and angular momentum . If the Hilbert space does not contain the spin space, the observables $\hat H$ (Hamiltonian), $\hat{ \mathbf{L}}^2$ and $\hat L_z$ form a complete set of compatible variables. These variables commute and the quantum state $|\psi\rangle$ is defined, through the indication of the eigenvalues of those operators. The position and momentum operators do not commute with the angular momentum operator. So the position and the momentum of an electron in the quantum state $(nlm)$ cannot be measured sharply.
So what does $\psi(\mathbf{x}) = \langle \mathbf{x} | \psi \rangle $ mean? One can show that $\{ |\mathbf{x}\rangle \mid \mathbf{x} \in \mathbb R^3 \}$ is a basis of the orbital Hilbert space. So $|\mathbf{x}\rangle$ and $|\psi \rangle$ are both vectors of the Hilbertspace. And $\langle \mathbf{x}|\psi\rangle$ is the scalar product of these vectors. If you want to charaterize a vector $\mathbf{r}$, you do this with the coordinates (x,y,z):
$$
\mathbf{r} = x \hat{\mathbf{e}}_x + y \hat{\mathbf{e}}_y + z \hat{\mathbf{e}}_z.
$$
For example $x = \hat{\mathbf{e}}_x \cdot \mathbf{r}$.
So $\langle \mathbf{x}|\psi\rangle$ are the coordinates of $|\psi \rangle$, related to the basis $\{ |\mathbf{x}\rangle \mid \mathbf{x} \in \mathbb R^3 \}$. You can express $|\psi \rangle$ with the basis vectors
$$
|\psi \rangle = \int_{\mathbb{R}^3} \mathrm{d}^3x |\mathbf{x}\rangle\langle \mathbf{x}|\psi\rangle = \int_{\mathbb{R}^3} \mathrm{d}^3x |\mathbf{x}\rangle\psi(\mathbf{x}).
$$
If you think of an integral, beeing a sum ober infinitesimal steps, you see that this is analogue to the cooridnate representation of $\mathbf{r} $.
The probability intepretation in of the wave function is the following:
$$
|\psi(\mathbf{x})|^2
$$
is the probability density for finding the quantum system at position $\mathbf{x}$, i.e.
$$
\int_V \mathrm{d}^3 x |\psi(\mathbf{x})|^2
$$
is the probability to find the system in $V \subset \mathbb R^3$.