As existing theories are replaced by new theories the existing words are redefined.
During the time that Caloric theory was assumed to be true the word 'heat' was understood as 'amount of Caloric'. Some time later Caloric theory was replaced by the kinetic theory of heat. Physicists did not stop using the word 'heat'. Instead 'heat' was redefined: heat is kinetic energy of molecules and atoms.
The newtonian definition of 'force' is: that which causes acceleration with respect to the local inertial coordinate system.
So: upon moving from newtonian dynamics to GR we need to redefine how the word 'force' is to be understood.
One way is to define 'force' as the derivative over distance of potential energy. We all agree that gravitational potential energy is an indispensible concept. Example: as a comet moves from the outer reaches of the solar system to the inner planets gravitational potential energy is converted to kinetic energy.
If it is agreed that there is such a thing as gravitational potential energy then it follows that gravity is a force.
When people say: "Gravity is not a force" then what they are actually saying is:
"If we transition to general relativity, but at the same time we keep using the newtonian definition of force, then gravity is not a force."
[Later edit](currently this answer is at 3 upvotes)
An example of a phenomenon that is not categorized as a force is degeneracy pressure. As we know: the importance of degeneracy pressure cannot be overstated. It defines the Chandrasekhar limit, etc., etc.
Degeneracy pressure plays a fundamental role, but: the concept of potential energy, and conversion of potential energy to kinetic energy, is not applicable in the case of degeneracy pressure. Just that is already sufficient to put degeneracy pressure outside the category 'force'.
It seems: what people are keen to express is that the mediator of gravitational interaction is very different from the respective mediators of the "other three". I assume people feel a need to express that gravitational interaction appears to be in a category of its own. The view that gravity is in a category of its own is good physics; it's just that the assertion "Gravity is not a force" is the wrong vehicle for expressing such a view.
[Later later edit]
About degeneracy pressure.
Let me make a comparison. Geophysicists have performed experiments where a tiny amount of a particular substance is subjected to the kind of pressure that otherwise exists only thousands of kilometers deep in the Earth. (This type of setup is called a 'diamond anvil cell'.) At such high presssure it is observed that substances are in a denser crystal structure than at surface pressure.
That denser crystal structure is unstable at surface pressure because the denser structure is like a highly compressed (coil) spring. It's a higher state of energy than at surface pressure, but at the extreme pressure deep in the Earth it is the lowest available state of energy.
The most extreme state for ordinary matter is the state of energy of the matter that constitutes a white dwarf. Specifically, a white dwarf that is close to the Chandrasekhar limit.
The matter constituting the white dwarf is extremely compressed, putting it in the highest possible state of energy. If some more matter accretes onto that white dwarf, raising the mass above the Chandrasekhar limit, then there is a most profound transition. No longer can the matter climb to an even higher state of energy: that path is exhausted. Then the entire composition of the white dwarf transitions to a neutron star.
To be a neutron star is a higher state of energy than before the transition to being an neutron star.