I'd like to know the exact context in which Paul Dirac introduced the Dirac delta function. What was the physical significance of the Dirac delta function when he first used it in Physics ?
The delta function was used to give the x-operator eigenstates in quantum mechanics. The original papers are relatively obscure, because all the material in them was incorporated into the Principles of Quantum Mechanics, which is in print and very popular. The first mathematical chapters contains a bunch of delta-function identities.
Plane-wave states are of the form $\exp(ikx)$, and these are clearly a useful quasi-basis for continuous functions. But when you Fourier transform, their x-fourier transform is a delta functions at position $k$. So if you want a representation of wavefunctions which is symmetric between x-space and p-space, you are forced to consider delta functions.