The Berry's or Zak's phase is given as
 
\begin{align*}
\gamma & =\oint_{BZ}d\mathbf{k}\mathcal{\mathcal{\mathcal{A}}}(\mathbf{k})\ \ \mbox{mod }2\pi\\
 & =i\oint_{BZ}d\mathbf{k}\langle u_{n}(\mathbf{k})|\nabla_{\mathbf{k}}u_{n}(\mathbf{k})\rangle\ \ \mbox{mod }2\pi\\
\end{align*}



 which can take any value between  $[0,2\pi]$.

 Now we want to learn the implications of the symmetries on this Berry's
 phase first assume that our system has only inversion symmetry such that
 the hamiltonian obeys



$U_{p}H(\mathbf{k})=H(-\mathbf{k})U_{p}$




then we have 
\begin{align*}
U_{p}H(\mathbf{k})U(\mathbf{k}) & =E(\mathbf{k})U_{p}U(\mathbf{k})\\
H(-\mathbf{k})U_{p}U(\mathbf{k}) & =E(\mathbf{k})U_{p}U(\mathbf{k})\\
H(\mathbf{k})U_{p}U(-\mathbf{k}) & =E(-\mathbf{k})U_{p}U(-\mathbf{k})
\end{align*}


such that $E(k)=E(-k)$ and  $U_{p}U(-\mathbf{k})=e^{i\phi(\mathbf{k})}U(\mathbf{k})$ where $U_{k}$ is the vector representation of  $|u(k)\rangle$.
 Now the Berry connection is

\begin{align*}
\mathcal{A}(\mathbf{k}) & =i(U_{p}U(-\mathbf{k})e^{-i\phi(\mathbf{k})})^{\dagger}\nabla_{\mathbf{k}}e^{-i\phi(\mathbf{k})}U_{P}U(-\mathbf{k})\\
 & =i(U(-\mathbf{k}))^{\dagger}e^{i\phi(\mathbf{k})}\nabla_{\mathbf{k}}e^{-i\phi(\mathbf{k})}U(-\mathbf{k})\\
 & =-\mathcal{A}(-\mathbf{k})+\nabla_{\mathbf{k}}\phi(\mathbf{k})
\end{align*}




 and the Berry's phase is for a 1d  $BZ$ for simplicity

\begin{align}
\gamma & =\intop_{-\pi}^{\pi}dk\,\mathcal{A}(k)\\
 & =-\intop_{-\pi}^{\pi}dk\,[\mathcal{A}(-k)+\nabla_{k}\phi(k)]\\
 & =\left.\phi(k)\right|_{-\pi}^{\pi}-\underbrace{\intop_{-\pi}^{\pi}dk\,\mathcal{A}(k)}_{\gamma}\label{eq:-18}\\
 & =2\pi n-\gamma\label{eq:-17}
\end{align}



thus we have 
\begin{align*}
\gamma=\pi n
\end{align*}



since $\gamma$ is well defined only up to $\mbox{mod }2\pi$ it can be $0$ or $\pi$ while obeying the last condition.
My suspission arises from the last step. Berry phase should be gauge independent but here it looks like gauge term determines its value.