A reference frame is simply a co-ordinate system - a self-consistent rule for assigning a unique set of four co-ordinates to every event in the universe. If Newton’s first law applies when co-ordinates of events are expressed in that reference frame (without needing to introduce artificial forces such as centrifugal force) then we call it an inertial reference frame.
Objects can move however they like in an inertial reference frame - we can have both accelerating objects and non-accelerating objects (relative to the reference frame). However, the net force on an accelerating object must be non-zero, and the net force on a non-accelerating object must be zero, so that Newton’s first law applies.
To qualify as an inertial reference frame, this property of accelerating and non-accelerating objects must be true at all times and for all objects. A reference frame is therefore always either inertial or non-inertial; it cannot change from one to the other.
We often attach a reference frame to an object by defining the position of that object to be the origin of the spatial co-ordinates in that reference frame at all times (we also need to define an orientation for the spatial co-ordinates and a rule for assigning a time co-ordinate value). By definition, an object is stationary in its own reference frame, and so is not accelerating in that reference frame. The reference frame attached to an object cannot therefore be an inertial reference frame if the net force acting on that object is not zero. Note that the converse is not true - the reference frame attached to an object can be non-inertial even if there are no forces acting on the object (the object could be rotating, for example).
One implication of this is that if two objects are accelerating relative to one another, then the reference frames attached to those two objects cannot both be inertial - one or possibly both of the reference frames will be non-inertial.