This is my first question on the Physics Stack Exchange; I hope it is not be a duplicate… :-|

I am not really a physicist (I am actually a professional mathematician), but I have already read quite a few things about quantum computation. And there is a problem that puzzles me: in quantum algorithms, it is assumed that you apply certain quantum gates like, say, the Hadamard gate:

$$H = \frac{1}{\sqrt{2}} \begin{pmatrix}1 & 1\\ 1 & -1\end{pmatrix}.$$

My point is: I do not doubt that you can apply some quantum transformations that are very close to the Hadamard gate, say

$$\tilde{H} = \frac{1}{1.414215} \begin{pmatrix}0.999999 & 1.000001\\ 1.000001 & -1.000002\end{pmatrix};$$

but getting a perfectly precise Hadamard gate seems as impossible to me as, say, building a corner reflector with perfectly right angles between the three mirrors (note that even defining what the angles are worth is meaningless below a certain precision level!). Moreover, as far as I understand it, certain quantum algorithms (including the Shor factorization) require so many successive applications of the Hadamard gate that even a $10^{-100}$ error in building such a gate would ruin the result of the algorithm…

The fact that surprises me most is that I have never read anywhere considerations on that issue of the precision of operators!—I have found much stuff devoted to the problem of interactions with the outside and decoherence, but nothing about the problem that I am concerned with…

Can anyone help me? Am I missing some fact which makes the precision problem actually irrelevant? Or, if that problem is relevant, are there some references about this issue? Maybe a classification of how much different algorithms would be sensitive to that matter…?