I'm having the same doubts as the poster of this question. Even after going through the logic of the answer, I still don't get how it resolves the issue of having the factor of $N$ on the first term, not the second term in AS:
Upon doing the first order correction to the Green's function (which includes the terms with one interaction that enter the first order correction of the self energy), it's pretty clear that the two contributions to $\langle\sum_{c,d,\mathbf{p_{1,2,3}}}\phi^a_{\mathbf{p}}\phi^b_{\mathbf{-p}}\phi^c_{\mathbf{p_1}}\phi^c_{\mathbf{p_2}}\phi^d_{\mathbf{p_3}}\phi^d_{-\mathbf{p_1}-\mathbf{p_2}-\mathbf{p_3}}\rangle$ have the following diagrams:
Now, the first diagram is proportional to $\sum_{c,d}\left(G^{ac}_{0,\mathbf{p}}G^{cb}_{0,\mathbf{p}}\sum_{\mathbf{p'}}G^{dd}_\mathbf{p'}\right)$ while the second diagram is proportional to $\sum_{c,d}\left(G^{ac}_{0,\mathbf{p}}G^{bd}_{0,\mathbf{p}}\sum_{\mathbf{p'}}G^{cd}_\mathbf{p'}\right)$ (note that shifted the momentum sum to account for the p-p' in the text - which is possible by translational invariance of the $\phi^4$ action). We can read off the first order correction to the self energy from this, from which it is clear that we get a factor of $N$ from the sum over d in the first diagram! This is not what AS claims in the equation above. The answer in the original post suggests that this factor of $N$ can be absorbed into the definition of feynman rules in this case, but I don't buy it. Regardless of definitions, $N$ is still a parameter that controls the non-crossing approximation - the first term is in higher order of $N$ than the second term regardless of the feynman rule applied. The only explanation now is that my calculations are incorrect (or I have a serious misunderstanding of how feynman diagrams work). Please help me. I've been stuck for a whole week.
EDIT Making a brave claim here: I think Altland Simons is just wrong in this section. As suggested in the stackexchange post I linked above, it should be clear that complete boson loops do indeed give a factor of N (and the second loop should indeed dominate in the large N limit). It's also in what appears to be fradkin's notes. So would the NCA involve "tree like diagrams" only? But this isn't what I see in some papers. I'm still confused, but I definitely see contradictions between reputable sources here.