I have just begun studying general relativity and have a question.
I know that if a tensor is zero in one coordinate system, it will be zero in all coordinate systems.
So how can the $\mu$'th component of the velocity $dx^\mu/d\tau$ be a tensor? There must be something I have misunderstood.
The three-velocity isn't a tensor - it's the four-velocity that is: this is a four-vector whose spatial components are the three-velocity of the particle (with times measured in the proper time of the particle), but which also has a zeroth (temporal) component $$ \frac{\mathrm d x^0}{\mathrm d\tau} = \frac{\mathrm d t}{\mathrm d\tau}, $$ which basically measures the rate of time dilation between your chosen frame of reference and the rest frame of the particle. No Lorentz transformation can transform a four-vector with a nonzero temporal component into one with a vanishing temporal component (and, moreover, no orthochronous Lorentz transformation can change the sign of the temporal component of a four-vector, so generally $\frac{\mathrm d x^0}{\mathrm d\tau}>0$).
This means that the four-velocity of any particle will always be nonzero: you can always set the spatial components to zero by transforming to the rest-frame of the particle, but if you do that then the temporal component will be $$ \frac{\mathrm d x^0}{\mathrm d\tau} = \frac{\mathrm d t}{\mathrm d\tau} = \frac{\mathrm d \tau }{\mathrm d\tau} = 1, $$ and the four-vector will not vanish.
velocity should not be a tensor
Why not? 4-velocity is a tensor: a $1 \choose 0$ tensor. It is the tangent vector to the worldline of a material particle, parametrized by proper time. Its components obey an identity: $$g_{\mu\nu} u^\mu u^\nu = 1$$ ($g_{\mu\nu}$ the metric tensor).
(I beg your pardon: there are two sign conventions here. I used the one I like better, but am not sure it is the one you are accustomed to.)
Also note that in GR you are widely free in the choice of coordinates (subject to some constraints I shall not dwell upon). In general you should not expect that coordinates themselves behave as components of a 4-vector. Still stranger as it may appear, you are not allowed (in general) to assign to one of the coordinates the character of a time and to the other three the one of space coordinates.
In other words, a coordinate line (e.g. $x^1=\rm const.$, $x^2=\rm const.$, $x^3=\rm const.$) is not bound to have a timelike tangent vector and the other three spacelike. All that is required is that the four vectors are independent.
Things are easier with orthogonal coordinates: then the metric tensor has $\ne0$ only the diagonal components, and in this case it is true that one of the coordinate lines is timelike and the remaining three spacelike. In many important cases such a choice of coordinates is possible, but not always. A counterexample is Kerr (rotating, uncharged) black hole.
In your question, you ask "how can the $\mu$-th component of $\frac{d x^{\mu}}{d \tau}$ be a tensor?"
To put it simply, it is not a tensor. The thing that is actually the tensor is the four-velocity $v$. The numbers $\frac{d v^{\mu}}{d \tau}$ are the components of this tensor in some particular coordinate system $x^{\mu}$. This could be Cartesian, spherical, etc.
You are correct that if you have an equation that sets the components of a tensor equal to zero, i.e. $$ A^{\mu\nu} = 0, \text{ for a rank (2,0) tensor, or } B^\mu = 0 \text{ for a rank (1,0) tensor,} $$
then these components are zero in every coordinate system. However, for the four-velocity there is no such true equation. You can't write $dx^{\mu}/d\tau = 0$, because even if the object is at rest (i.e. it has zero 3-velocity $\vec{v}$ ), the four velocity will be $$ v^\mu = \begin{cases}c & \mu = 0 \\ 0 & \mu = 1 \\ 0 & \mu = 2 \\ 0 & \mu = 3 \end{cases} $$
(This is not typical notation, usually people say would $v^\mu = (c,0,0,0)$, but for clarity I write it as a statement for each value of $\mu$). This is because the four velocity generally looks like this in Cartesian coordinates: $$ v^\mu = \begin{cases} \gamma c & \mu = 0 \\ \gamma v_x & \mu = 1 \\ \gamma v_y & \mu = 2 \\ \gamma v_z & \mu = 3 \end{cases}, $$
with $\gamma = \left(\sqrt{1-|\vec{v}|^2/c^2}\right)^{-1}$ the Lorentz factor, and $v_x, v_y, v_z$ the usual components of the 3-velocity in Cartesian coordinates.
Thus the four-velocity can never be entirely zero.
