In the traditional Physics literature, we find $\delta$ be defined as
$$\delta(x)=\begin{cases}0, & x\neq 0, \\ \infty, & x = 0\end{cases}$$
and then one "derives" that if $f$ is a function we have
$$\int_{-\infty}^\infty f(x)\delta(x-a)dx=f(a).$$
The problem is that $\delta$ doesn't make sense as a function. Indeed one can show that if one tries to define $\delta$ by this integral, there is no function that will make this equation true.
It turns out that this can be handled by the introduction of the theory of distributions. The idea is: define $\delta$ as something that acts on a function, giving the result of the above integral. In other words: make $\delta$ a linear functional over some functions.
The appropriate setting, at first, to do this, is to consider $\mathcal{D}(\mathbb{R})$ the so-called space of test functions, being those that are $C^\infty$ and vanish outside a closed and bounded interval. Consider now $\mathcal{D}'(\mathbb{R})$ the space of all continuous linear functionals over $\mathcal{D}(\mathbb{R})$. We denote the action of $\xi\in \mathcal{D}'(\mathbb{R})$ on the test function $f$ by $\xi[f]=\langle \xi,f\rangle$.
This means this is the space of all $\xi : \mathcal{D}(\mathbb{R})\to \mathbb{R}$ such that given a sequence of test functions $f_n\in \mathcal{D}(\mathbb{R})$ convergent to $f$, the sequence of numbers $\langle \xi,f_n\rangle$ converges to $\langle \xi,f\rangle$.
Now the Dirac Delta is defined as the distribution $\langle \delta, f\rangle = f(0)$ and the Dirac Delta centered at $a\in \mathbb{R}$ is $\langle \delta_a, f\rangle = f(a)$. It turns out that $\delta$ is what traditional literature calls $\delta(x)$ and $\delta_a$ is what is called $\delta(x-a)$.
The fact is that functions define distributions by means of taking the integral against the function. Also most operations you do on functions like differentiation and taking the Fourier transform, can be lifted to distributions, such that they coincide with the usual notions when restricted to functions. Thus distributions are sometimes called "generalized functions", even though they are not functions.
As you see $\delta(0)$ is not defined. Actually $\delta(x)$ is not defined, because $\delta$ isn't a function in the traditional sense, it doesn't act on numbers. It acts on functions. The point is that $\delta(\lambda - \lambda) = \delta(0)$ is so undefined as $0/0$. It is not infinite, it is just something that doesn't make sense. When you see $\delta(x)$ or $\delta(x-a)$ it is just a notation, to an object that just makes sense when applied to a function.