In thermodynamics, entropy (differences) is (are) defined, at least at first, only for two states which can be connected by a reversible transformation. Hence, those two states have to be equilibrium states. Yet, sooner or later, one always runs into the example of two boxes of gasses, at different temperatures, separated by an insulating wall. The entropy of this joint system is calculated by calculating the entropy of each box separately (each box is indeed an equilibrium state) and adding the results (and forgetting to specify the arbitrary additive constants needed, but never mind...). Then the wall is removed, and an irreversible path (this had better be studied using a ppVV diagram, one p and one V for each box, so that each one can have its own $T$...this is in line with the use of a pressure field) is followed from the now non-equilibrium state of each box's being at a different temperature to the equilibrium state of a temperature for the new joint system. Then its entropy is calculated and shown to be higher than before. Yet the entropy difference between the non-equilibrium state and this state is undefined according to the usual difference.
I was just looking at a paper by someone in a math dept. about the thermodynamics of power plants in which he considered a power plant in thermal contact with two atmospheres: the terrestrial one and the martian one. (No I am not making this up.) This inspired the following observation: what is an irreversible transformation in one setting can be considered a reversible transformation in a different setting.
Change the external conditions of the above systems. Each box is now in contact with a separate heat reservoir, at the different initial temperatures we considered. Insulating walls can be present or absent here, too. Now our thermodynamic state space has to have two p variables and two T variables, one for each box. Initially we have each box in contact with its reservoir. We break that contact and put the boxes in contact with each other infinitesimally short period of time, then break that contact (this is analogous to the infinitesimal addition of a weight to the piston). We have moved, reversibly, from one equilibrium state to another. (This is reversible because we could restore contact with the corresponding reservoirs and get back to where we started.) We do this again and again until finally the boxes have equalised their temperatures. This is a reversible transformation. On the ppVV diagram it looks identical to the irreversible transformation we discussed above. But the physics is completely different, because the latter was in an isolated system and this one is not in an isolated system.
Well, this is part of the answer to your question, too. The notions of equilibrium and reversibility depend on the contact with the environment which is supposed.
Now suppose an isolated system of the simplest kind, a perfect gas. Obviously $T$ cannot change. Can $V$ change? Suppose the mass is finite. If the gas is not enclosed in perfectly fixed, insulating walls, then obviously it will dissipate to infinity and we have $p=0$. There are no equilibrium states... So suppose $V$ is fixed. The first law of thermo still implies $T$ cannot change..not even irreversibly. But then the gas law implies $p$ cannot change, so there are no paths at all, neither reversible nor irreversible. All states are equilibrium states, but none of their entropies can be compared. The same holds if the mass is infinite but there is a constant density.
These two extreme examples show that the notions of equilibrium and reversibility depend on the external conditions. Since the science of Thermodynamics was inspired by heat engines (um, terrestrial ones), the usual analytical framework involves putting it in contact with a heat bath, breaking contact, putting it in contact again, etc., as in the Carnot cycle. Fundamental Theoretical Physics finds the closed conservative system more congenial, but such an isolated system looks really strange to Thermodynamics. Perhaps this is what is responsible for the pedagogical confusion students have with entropy.
