What happens to the $U(1)$ factor in the $U(N)$ SYM gauge group of the AdS/CFT correspondence? I'm learning about the AdS/CFT correspondence. I know that from the open string perspective, the dynamics on a stack of $N$ coincident $D3$-branes is given by a $\mathcal{N} = 4$ Super Yang Mills theory with gauge group $U(N)$ in four spacetime dimensions. 
However, as I understand it, the weak form of AdS/CFT relates $\mathcal{N} = 4$ SYM with gauge group $SU(N)$ to supergravity in ten dimensions on $AdS_5 \times S^5$.
I was confused since the gauge groups $U(N) \neq SU(N)$? How is this resolved then? Is this not a problem? 
 A: In this Review, section 3.1 page 58, there is a paragraph that answers your question:

"A $U(N)$ gauge theory is essentially equivalent to a free $U(1)$ vector multiplet times
  an $SU(N)$ gauge theory, up to some $Z_N$ identifications (which affect only global issues). In the dual string theory all modes interact with gravity, so there are no decoupled
  modes. Therefore, the bulk $AdS$ theory is describing the $SU(N)$ part of the gauge
  theory. In fact we were not precise when we said that there were two sets of excitations
  at low energies, the excitations in the asymptotic flat space and the excitations in
  the near horizon region. There are also some zero modes which live in the region
  connecting the “throat” (the near horizon region) with the bulk, which correspond to
  the $U(1)$ degrees of freedom mentioned above. The $U(1)$ vector supermultiplet includes
  six scalars which are related to the center of mass motion of all the branes [151]. From
  the $AdS$ point of view these zero modes live at the boundary, and it looks like we might
  or might not decide to include them in the $AdS$ theory. Depending on this choice we
  could have a correspondence to an $SU(N)$ or a $U(N)$ theory. The $U(1)$ center of mass
  degree of freedom is related to the topological theory of $B$-fields on $AdS$ [152]; if one
  imposes local boundary conditions for these $B$-fields at the boundary of $AdS$ one finds
  a $U(1)$ gauge field living at the boundary [153], as is familiar in Chern-Simons theories
  [25, 154]. These modes living at the boundary are sometimes called singletons (or
  doubletons)"

As you can see it is up to you to include the $U(1)$ part in the $SYM$ side of the duality. This choice is related to the choice of including or not local boundary conditions to the $B$-field in the $AdS$ side of the duality. See also the reference inside this Review, like this one
