On the physical interpretation of Dirac delta distribution The original purpose of Dirac was to make up the eigenstate of the position operator $\hat X$.
Now, quantum states are complex-valued functions in the Schwartz space $\mathscr{S}(\mathbb{R}^n)$
The main point is... $\delta$ is not a function. It can't be associated with a (unique) norm, since Schwartz space is just a Fréchet space, not Banach.
Therefore the probabilistic interpretation of it as a wave function, $|\delta|^2$, is non-sense, and still at some point I will for sure see a particle somewhere.
So... what is the correct way to look at it?
 A: Operators with purely continuous spectra - such as $\hat X$ - do not have any eigenstates.  The $\delta$-distribution serves as a generalized eigenstate, providing a formalization of $|x\rangle$ and $\langle x|$ which are extremely useful computational tools, despite the fact that they do not themselves correspond to physically realizable states.

[...] at some point I will for sure see a particle somewhere.

This is reflected in the fact that the probability of measuring the particle  somewhere is $1$; this doesn't require the existence of position eigenstates.


Using Born's rule the probability of obtaining the result $a$ from observable $A$ over a state $\psi$ is $\langle \psi|P_a|\psi\rangle$, where $P_a$ projects in the $a$-eigenspace of $A$. But in this case there is no eigenspace, for there are no proper eigenstates, so... ?

The map $\mu_A$, which associates a measurement outcome to the corresponding projection operator, is called a projection-valued measure.  By measurement outcome, I mean a (Borel-measurable) subset $E\subseteq \mathbb R$ which could consist of a single value - e.g. $\mu_A(\{a\})= P_a$, in your notation - or multiple distinct values, e.g. $\mu_A(\{a,b,c\}) = P_a+P_b+P_c$.
If the spectrum of the operator $A$ is purely continuous, then the projection $\mu_A(\{a\})=0$ for any individual point $a\in \mathbb R$.  Case in point, the PVM associated to the position operator is simply
$$\mu_X(E) = \chi_E \mathbb I \qquad \chi_E(x)= \begin{cases} 1 & x\in E \\ 0 & \text{else}\end{cases}$$
where $\chi_E$ is the indicator function on $E$ and $\mathbb I$ is the identity operator.  As a result, if the state of the system corresponds to some wavefunction $\psi\in L^2(\mathbb R)$, the probability of measuring $\hat X$ to lie in $E$ is given by
$$\mathrm{Prob}_\psi(\hat X,E) := \frac{\langle \psi|\mu_X(E)|\psi\rangle}{\langle\psi|\psi\rangle} = \frac{\int_\mathbb R \psi^*(x) \chi_E(x) \psi(x) \mathrm dx}{\int_\mathbb R \psi^*(x)\psi(x)\mathrm dx}$$
$$ = \frac{\int_E |\psi(x)|^2 \mathrm dx}{\int_\mathbb R |\psi(x)|^2\mathrm dx}$$
