A small issue in renormalisation group formalism In the general RG formalism, suppose $\vec{\mu}$ represents a vector in parameter space and $\vec{\mu}^*$ is the fixed point under the transformation $R$. Then for $\vec{\mu}=\vec{\mu}^*+\delta \vec{\mu}$, we have
$$R(\vec{\mu})=R(\vec{\mu}^*+\delta \vec{\mu})=\vec{\mu}^*+R'(\vec{\mu}^*)\delta \vec{\mu}$$
after which the small deviation $\delta \vec{\mu}$ is expanded as a linear combination of the eigenvectors of the matrix $R'(\vec{\mu}^*)$, denoted as $\{\vec{v}_i\}$ with $\{\lambda_i\}$ as the corresponding eigenvalues.
It seems that people normally just assume that the eigenvectors of $R'(\vec{\mu}^*)$ form a complete basis, which is obviously not true in general. Now let's assume that $\{\vec{v}_i\}$ does not form a complete basis, then we would have 
$$\delta \vec{\mu}=\sum_i c_i\vec{v}_i+\Delta \vec{\mu}$$
where $\Delta \vec{\mu}=\delta \vec{\mu}-\sum_i c_i\vec{v}_i$ and $\Delta \vec{\mu} \perp \vec{v}_i$ for all $i$. This simply means that $\Delta \vec{\mu}$ is the projection of $\delta \vec{\mu}$ onto the normal of the hyperplane spanned by $\{\vec{v}_i\}$. In this case, the linear transformation from the linearisation obviously keeps $\Delta \vec{\mu}$ invariant. Thus, if $\lambda_i <0$ for all $i$, we will get $\vec{\mu}^* +\Delta \vec{\mu}$ in the end, which effectively seems to be a new shifted fixed point. Indeed, we have
$$R(\vec{\mu}^*+\Delta \vec{\mu})=\vec{\mu}^*+R'(\vec{\mu}^*)\Delta \vec{\mu}=\vec{\mu}^*+\Delta \vec{\mu}$$
Since $|\Delta \vec{\mu}|<<1$, it seems that the two fixed points are very close to each other and in fact they can be made arbitrarily close if the the $\delta \vec{\mu}$ is carefully chosen. This seems to be a quite strange result. Furthermore, along the way we only kept the linear terms of the expansions, whereas I suspect that higher order terms may play a role. 
The overall picture seems rather blurred to me. Really appreciate if someone can help clarify.
EDIT: Thanks to @Brightsun, $\Delta \vec{\mu}$ is not necessarily invariant under the action of $R'(\vec{\mu}^*)$ (example given in the comment)
 A: The assumption of a full eigensystem is usually made for convenience. But it is not always satisfied. If it is not satisfies one gets additional logarithmic contributions to the scaling laws. This is discussed, e.g., in the paper by Wegner and Riedel.
A: A typical renormalization group flow can be thought of as a smooth vector field $\vec V(\mu)$ defined on parameter space. Starting with parameters $\vec\mu(\ell)$ at scale $\ell$, you obtain parameters at scale $\ell'$ by solving the differential equation $\frac{d\vec\mu}{d\ell}=\vec V(\vec\mu(\ell))$. The function $R$ referred to above can be thought of as the process of solving this equation over some change in length scale.
Fixed points of RG flow (scale-invariant field theories) then correspond to points $\vec \mu^*$ where $\vec V(\vec \mu^*)=\vec 0$.  Assuming $\vec V(\vec \mu)$ is smooth, we can expand near $\vec\mu^*$ (this information we can usually get from field theoretic methods like the $\epsilon$-expansion in the neighborhood of simple `Gaussian' fixed points). 
The situation you describe above (where $R(\mu^*+\Delta\mu)=\mu^*+\Delta\mu$ for a continuum of values of $\Delta\mu$) corresponds to the case when the zeros of $V(\mu)$ are not always isolated points, but curves or even hypersurfaces in parameter space.  
A trivial example where this happens is if some operator $K$ that is conventionally assigned scaling dimension 0 is rewritten as $K=K_1+K_2$ in an enlarged parameter space. Sometimes there is a physical motivation to enlarge parameter space this way, like when there is an anisotropic crystal with a direction-dependent speed of sound. For example, if anisotropy breaks an $SO(2)$ symmetry to a $\mathbb{Z}_2$ symmetry, the system will behave like an Ising model for a large part of parameter space, behaving like an $SO(2)$ model only for highly ungeneric parameter values.  
For intuition about the eigenspaces of $R'(\mu^*)$, it may help to look up graphs of various real RG flows (like in superconductors, or liquid crystals) and see how the flows behave near fixed points.
