Consider two hypothetical nuclei both with the mass number $A=6$ but one with $Z=N=3$ and the other with $Z=2, N=4$. Let us assume that each nucleon moves in an average nuclear potential as in the shell model, which is a simple harmonic potential equally spaced levels having energies $E_0, E_0+\delta, E_0+2\delta,...$, etc.
In the case of the 1st nucleus, two protons (spin-up and spin-down) and two neutrons (spin-up and spin-down) will go into the ground state, and the remaining one proton and one neutron will go into the first excited state. Therefore, its energy will be $4E_0+2(E_0+\delta)$.
In the case of the 2nd nucleus, two protons (spin-up and spin-down) and two neutrons (spin-up and spin-down) will go into the ground state, and the remaining two neutrons will go into the first excited state. Therefore, its energy will again be $4E_0+2(E_0+\delta)$.
But don't we expect that the second nucleus to have higher asymmetry energy due to asymmetry in the neutrons and proton numbers? Forget about pairing energy and all that. Why isn't the asymmetry energy making any contribution here or is my caricature of the shell model too naive to capture this? Isn't $N\neq Z$ supposed to make an additional contribution?