I will give a semi mathematical point of view, and a completely intuitive point of view.

**Mathematical(ish) POV**

We have $$\langle \alpha |A|\alpha\rangle = 0 \  \ \forall |\alpha\rangle$$

Switching to the matrix view of operators we can say 

$$A = \sum \limits_n a_n |a_n\rangle\langle a_n|$$




Thus
$$|a_n|^2 ||\langle \alpha|a_n\rangle||^2 = 0 $$

Besides the trivial case the inner product will not be 0, but we also have the condition from the axioms of QM that $|a_n|^2 \geq 0 $. Thus the only way for this to be true for an arbitrary $|\alpha\rangle$ is for the coefficients to be $0$ which is equivalent to saying $A = 0$.

(Another way of seeing this is since $|\alpha \rangle$ is arbitrary, choose it to be one of the eigenvectors, $|a_n\rangle$. Each time you do this the only solution is that the coefficient is 0. Repeat $\forall n$ and you'll see that $A$ must be the $0$ operator.