TL;DR Lorentz contraction is towards the moving origin.
I think your confusion is best clarified in the simpler situation of inertial frames$^{[0]}$, which I consider first. (For the case of accelerated frames, I'll append later)
The entire space$^{[1]}$ of a moving reference frame appears Lorentz contracted towards the origin of that reference frame, to stationary observers. In other words, the origin of the moving reference frame is the fixed point under Lorentz contractions (LC). There is nothing special about relativity in this regard - whenever a scaling transform is applied to a vector space, its origin remains unchanged.
fig1: A marked ruler as it appears in its rest frame $S'$ (top) and as it appears to observers with respect to whom it is moving with uniform constant velocity (frame $S$, bottom). The appearances have been aligned so that the origins coincide. Clearly, the ruler appears contracted towards the origin ($\gamma=1.2$).
Elaborating further, when we talk of the LC of a moving rod, what we really mean is the length between the simultaneous measurements of the positions of its ends (by stationary observers). The fact that this length is smaller (wrt. its value in its rest frame) doesn't imply that one end is contracting towards the other as the other end remains fixed. As you point out, this is ambiguous. Instead distances of both ends measured from the origin, contract towards the origin, by the same factor, and so does their difference. Indeed, every point of the rod, nay, the entire space$^{[1]}$ contracts towards the origin. Since the contraction is uniform, there is no fracture.
Besides the last line, the lack of fracture is expected for a more foundational reason: simply by changing your frame of reference, an object can't suddenly be made to appear fractured - after all, in its rest frame there are no forces acting on it, and so it sits unchanged for all time; this physical reality of the rod is identically observed by all observers - inertial or not.
Is there anything special about the origin of the moving frame? Absolutely not. The fixed point is determined by the initial alignment of the origins of the two frames that are moving relative to each other, and the initial sync of their clocks. Indeed, with an appropriate choice, the origin can be arbitrarily shifted. This is why associating the quality of 'towardness' to scaling isn't really useful.
concretely
This skippable section puts the above explanation on a firmer footing. let $S'$ be a reference frame ('rod frame') moving with constant uniform velocity $\beta\hat x$$^{[2]}$ wrt. the frame $S$ ('lab/observer frame'). Further, by choice, let their origins coincide and axes align when their clocks are synchronized. Let some arbitrary event $E$ be described by the coordinates $(t',x')$ in $S'$ and $(t,x)$ in $S^{[3]}$. These are connected by a Lorentz transformation:
$$ \begin{align} x'&=\gamma(x-\beta t)\\ t'&=\gamma(t-\beta x) \end{align}\tag{1} $$ where $\gamma=(1-\beta^2)^{-1/2}$.
Consider the two events as described by observers in $S$ (for some $L>0$).
$$ \begin{align} E_0&:(t,\beta t)\\ E_1&:(t,\beta t\pm L)\tag{in $S;\>2$} \end{align} $$
As you may have guessed, $E_0$ is the spacetime coordinate of the origin of the moving frame, and $E_1$ an arbitrary point at distance $L$ (wrt. observers in $S$) from it. Note the same time coordinate: the events must be simultaneous for observers in $S$ to constitute a length measurement.
Observers in frame $S'$, on the other hand, describe the same events with coordinates
$$ \begin{align} E_0&:(t/\gamma,0)\\ E_1&:(t/\gamma\mp\beta\gamma L,\pm \gamma L)\tag{in $S';\>3$} \end{align}$$ Ignore the time coordinate (it's complicated because of time dilation). The space coordinate of $E_0$ is always $0$, which it must be since to observers in $S'$ it corresponds to their origin. Note the correct contraction: observers in $S'$ conclude that observers of $S$ are measuring a contracted $L$ between the two events instead of the 'correct' $\pm \gamma L$. Notice how the $\pm$ denotes the contraction towards the origin.
But $E_1$ could have referred to any point. So the length of all points measured from the point $\beta t$ at time $t$, appears contracted by $\gamma$ to the observers in $S$, in the opinion of observers in $S'$. Indeed, as seen in eqns. $(1)$ for the Lorentz transformation, the contraction happens on the length $x-\beta t$ i.e. length measured (by observers in $S$) from the origin of the moving ref. frame.
secondary
"Then, in comparison to its position if length contraction did not occur (but say stuff like time dilation and whatnot did, not sure what this would affect, let me know if this assumption is wherein lies the issue)"
If "length contraction did not occur" neither can time dilation. By definition, the non-contracted length is the length of the object as measured in its rest frame; wrt. this all other frames measure smaller lengths. Same goes for the time intervals. A time interval, by definition, is undilated in the rest frame of the object, wrt. which all other frames measure it to be larger.
footnotes
$^0$the rod's not accelerating but moving uniformly w.r.t. the observer.
$^1$ Components of vectors along the direction of relative motion. Orthogonal components aren't contracted.
$^2$$c=1$
$^3$ coordinates $y,z$ don't change and are suppressed