I just read on this webpage that we have (click me) $g_{\alpha \beta} = g_{\alpha}^{\beta} = g^{\alpha \beta}.$
Now, although I understand that the first and the last one are equal, I don't think that the term in the middle is the same as the other two, cause we should have $(g_{\alpha} ^{\beta}) = (g_{\alpha \alpha'})(g^{\alpha' \beta})$. This should be equal to the identity matrix.
What am I doing wrong?
That statement is nonsense.
While it is true, that, in flat space, the components of $g^{\mu\nu}$ and $g_{\mu\nu}$ are exactly the same, the equation $g^{\mu\nu} = g_{\mu\nu}$ is not a valid equation - the indices don't match.
As you correctly observe
$$ {g_\mu}^\nu = g_{\mu\rho}g^{\rho\nu} = {\delta_\mu}^\nu$$
since $g_{\mu\nu}$ is the inverse matrix of $g^{\mu\nu}$.
First a word on notation. In special relativity, the Minkowski metric is $\eta_{\alpha\beta}$. The general relativity curved metric is $g_{\mu\nu}$. A lot of texts that only use the Minkowski metric don't make that distinction for some reason. However, when you get to string theory and there are four different metrics floating around, it is important to keep things straight by calling $\eta_{\alpha\beta}$ the flat spacetime metric. This is a huge pet peeve of mine.
Second pet peeve: The statement that the co- and contravariant tensors are the same is rubbish. The components are the same. It doesn't even make sense to say that two tensors belonging to different tensor algebrae are equal.
Now to your question. That is an error on the author's part. You are completely correct. As you have shown, $$\eta^\alpha_\beta=\delta^\alpha_\beta$$
