This is a good question. In fact, as I'll explain, it leads right to some topics of current research.
First, an important fact about entropy: if you look carefully at the proof Kittel gives, it should only be applicable to a situation in which you add degrees of freedom to a small system that is exchanging energy with some heat bath. However, if you consider an isolated system in which you know as much information as possible about every part, the total entropy of this system is not necessarily larger than the entropy of a small part of the system. In fact, if you know everything possible about a closed system its entropy is zero, since you know the microstate, while a subsystem may have non-zero entropy.
Anyway, as you say, let's imagine a hypothetical universe that is roughly static, and has some region that is interacting. In addition, there is a far-off region that is nearly isolated, and only connected by some probe that allows it to watch the rest of the "universe" without significantly disturbing it. From the perspective of an observer in this far-off region, the rest of the universe is essentially closed, but still appears to be evolving toward equilibrium according to principles of thermodynamics. This is a bit of a puzzle. Thermodynamics is usually defined in an open system, especially for a system which can exchange energy with some exterior heat reservoir, and without this it is not clear, for example, what role entropy plays. This problem actually gets worse, if anything, when you try to describe your universe with quantum mechanics. Roughly speaking, in quantum mechanics you can never lose information about an isolated system, and losing information is necessary for entropy to increase.
You may be surprised to hear that the resolution to this question is actually not completely settled! However, a popular proposal goes by the name of the "Eigenstate Thermalization Hypothesis." This says, in essence, that some (but not all) interacting closed systems follow an evolution such that any small piece of the system sees the rest of the system as a heat bath, and as a result each part approaches a state that looks like an equilibrium. If this were true of the universe, it would mean that the entropy of the whole thing does not change, but if you look at any subset of the universe then the entropy of that subset always increases in the same way an open system would. As I mentioned at the top, this is possible because the definition of entropy is such that a subsystem can have a larger entropy than a total closed system.
As suggested by the word 'hypothesis,' this is more of a proposal than an ironclad explanation. However, we can actually do experiments that show that this happens in some cases. Ultracold atomic gas experiments can create an interacting system that is very, very close to completely isolated- it is levitated by electric and magnetic fields, in a nearly perfect vacuum, and too cold to emit any significant radiation. Nonetheless, if you disturb these systems out of equilibrium, and wait for awhile before you check them, in some cases they will return to what looks like a new thermodynamic equilibrium, in a sense appearing as if their entropy increased even though the system is closed. This shows that at least in some cases this kind of behavior is possible, but research is ongoing to try to understand under what circumstances it is true. For the universe as a whole, I think it is fair to say that all bets are off.