Does dividing a box full of gas at equilibrium change its entropy? Imagine you have a box full of a gas. It is at thermodynamic equilibrium.
The entropy of the gas in the box is proportional to the logarithm of the number of available microstates.
The number of microstates is proportional to the number of oscillators (abstractly speaking) times the number of available states per oscillator.
Let's limit ourselves to the translational case. In the ideal particle-in-box model, we know that the possible momenta of the particles is quantized, due to the boundary conditions imposed by the sides of the box. 
If we were to put a divider in the middle of the box, half of the possible momentum states would now be disallowed. Previously, wavefunctions corresponding to each particle were allowed to have an antinode in the middle of the box ("odd solutions") but now can only have wavefunctions with a node in the middle of the box ("even solution"). There are now fewer translational microstates, so entropy should be reduced.
However, as far as I know, putting a divider in a box does not change the entropy.
A possible solution here is that perfectly rigid dividers and boxes don't exist; a particle can impart momentum into the divider, which can then be imparted to particles on the other side, etc., which would increase the effective number of microstates. However, we should still expect some reduction in available microstates, and this cannot be a complete explanation because we know that if we have two different gasses separated by a divider, by removing the divider we increase total entropy. This, and the fact that increasing volume increases entropy, is nicely explained by the fact that there are more available momentum states in a larger box. However, I cannot reconcile this with the "divider inserted into an equilibrium gas" example I gave above.
Edit: The proposed solution does not address the question in terms of additional momentum state density. I wouldn't even say it actually answers the question at all. "because the thin membrane doesn't materially change the system and carries a tiny entropy by itself. " is not an explanation, just a tautological re-statement of the question.
 A: Things are complicated: The Statement "dividing the Box doesn't change entropy" is a statement that holds only in the thermodynamical limit, for various reasons: 
To get to the core at the start:
Entropy is definded to be additive, that means if you have two statistical systems with two Distributions $\rho_1$ and $\rho_2$, then the combined entropy of the two (still divided!) Systems is $S(\rho_1, \rho_2) = S_1(\rho_1) + S_2(\rho_2)$. BUT: Bringing the two Systems together will change the Systems, and thus Change the entropy on various ways:
Let's assume the case of a classical gas (although your question is asked about a quantum gas) divided into two volumes. 
Technically this is the situation of two microcanonical ensembles that aren't interacting, and thus each microstate has an Energy $H_1$ and $H_2$. At the moment you remove the divider, the whole Energy of a microstate of the whole System will be $H_1 + H_2 + H_i$. $H_i$ describes interactions between the gas particles of different origin. 
So you have another hamiltonian now, this changes the distributions of the two systems, and thus the entropy changes. 
With large Systems (thermodynamical limit), the interaction-term becomes small, compared to the other terms, and in case of an ideal gas, there is no interaction between any particles. 
Still, even with an ideal gas, entropy increases when you take away the divider: You allow the two volumes to exchange particles. You now don't have two microcanonical ensembles, but instead two canonical ensembles. This fact also changes the distribution of the possibles states, and thus increases entropy. Again, in the thermodynamical limit, all ensembles become aequivalent, and thus the change in entropy can be neglected. 
None of these examples meets your situation, but the arguments should be about the same. You simply have another System, wether there are two divided subsystems or one big System yields another hamiltonian. 
