You might see it from the notion of Fourier transformation. For example, you could express an arbitrary quantum state of your Hilbert space in momentum representation by applying a Fourier transformation on your position representation. More explicitly, for $\left|\psi\right>\in\mathcal{H}$, you might define
$$\left< \mathbf{q} |\psi\right> = \mathcal{F}^{-1} \left[\left< \mathbf{p} |\psi\right>\right] = \frac{1}{\mathcal{N}_F} \int_{\mathbb{R}^n}\left< \mathbf{p} |\psi\right> e^{i \mathbf{p}\cdot\mathbf{q}/\hbar} \text{d}p^n \hspace{2cm}(1)$$
Now we are almost there. You can make use of the fact that position and momentum fulfil completeness relations. For example you could have
$$\int_{\mathbb{R}^n} \left|\mathbf{p} \right> \left< \mathbf{p}\right| \text{d}p^n = \mathcal{N}_p $$
Inserting this on the left hand side of equation (1) gives
$$\left< \mathbf{q} |\psi\right> = \frac{1}{ \mathcal{N}_p} \int_{\mathbb{R}^n} \left< \mathbf{q} |\mathbf{p} \right> \left< \mathbf{p}|\psi\right> \text{d}p^n$$
Identifying this with the right hand side of equation (1) leads to the desired relation
$$\left< \mathbf{q} |\mathbf{p} \right> = \frac{\mathcal{N}_p} {\mathcal{N}_F} e^{i \mathbf{p}\cdot\mathbf{q}/\hbar} $$
The normalizations $\mathcal{N}_i$ depend upon conventions without general consensus. And by the way, you can also recover your representation of the delta function (which is inherent to the notion of Fourier transformation).
EDIT: To elaborate the connection to the delta function a little bit more. You might interpret $\delta$ as a functional acting on your Hilbert space. In other words you define that
$$\left< \mathbf{q} |\psi\right> = \int_{\mathbb{R}^n} \delta^{(n)}(\mathbf{x}-\mathbf{q})\psi(x) \text{d}x^n$$
where $\psi$ could be element of $\mathcal{L}(\mathbb{R}^n)$, for example. Then, for $\psi,\phi\in \mathcal{L}(\mathbb{R}^n)$ you use the standard definition of a scalar product on this space
$$\left< \phi |\psi\right> = \int_{\mathbb{R}^n} \phi^\ast(\mathbf{x}) \psi(\mathbf{x}) \text{d}x^n = \int_{\mathbb{R}^n} \left< \phi|\mathbf{x}\right>\left< \mathbf{x} |\psi\right> $$ and you see that we get the completeness relation
$$\int_{\mathbb{R}^n} \left|\mathbf{x} \right> \left< \mathbf{x}\right| \text{d}x^n = 1 $$
And to close the circle now, you can expand (in the position basis)
$$\left< \mathbf{x} |\psi\right> = \int_{\mathbb{R}^n} \left< \mathbf{x}|\mathbf{y} \right> \left< \mathbf{y}| \psi\right> \text{d}y^n $$
So you identify
$$\left< \mathbf{x}|\mathbf{y} \right> = \delta^{(n)} (\mathbf{x}-\mathbf{y})$$
I hope this illustrates a bit how fundamental the appearance of this delta function is. Not a coincidence :)