The units do add up in the example you provide:

You've correctly established that the delta distribution has the dimension of the inverse dimension of its argument. This can be seen in a variety of ways, e.g. by looking at $\int f(x)\delta(x-x_0) dx = f(x_0)$, where the LHS has to have whatever dimensions $f$ has.

With this, we look at the dimensions of $\lvert A \rvert^2 2\pi\hbar \;\delta(p_1-p_2)$:
$$
\left[\lvert A \rvert^2 2\pi\hbar\; \delta(p_1 - p_2) \right] = \textsf{L}^{-1} \left[ \hbar \delta(p_1-p_2) \right] = \textsf{L}^{-1} \frac{\left[ \hbar \right]}{[(p_1-p_2)]} = \textsf{L}^{-1} [x] = \textsf{L}^{-1} \textsf{L}^{1} = 1,
$$where I have used that $\frac{p}{\hbar}$ has to have inverse dimensions of $x$ to make the argument of the exponential dimensionless.

---
<sub>
NB: For $\psi_p(x)=A\exp(\frac{ipx}{\hbar})$, we want $\int \lvert \psi \rvert^2 dx$ to represent a probability, which is dimensionless. From this it follows that $A$ has to have dimensions $\textsf{L}^{-\frac12}$.
</sub>