Let $S$ and $S'$ be inertial frames moving at a relative velocity $v$ in the $x$-direction. Imagine sending observers to all points in each reference frame. The following rules hold:
(1) In a given frame, all observers agree to measure the spatial coordinates $ (x, y, z)$ of an event $P$ with respect to the origin of their reference frame.
(2) In a given frame, all observers' clocks are synchronized using some method (such as Einstein synchronization).
Consider two observers in the $S$-frame. Let observer $A$ be located at $(x, y, z) = (x_A,y_A,z_A)$. Let observer $B$ be located at $(x, y, z) = (x_B,y_B,z_B)$. Assume an event $P$ occurs at some other point in the $S$-frame. By rule (1), both observers $A$ and $B$ will agree on the spatial position of $P$, say $(x, y, z) =(x_P, y_P, z_P)$. However, since the speed of light is a finite constant in the inertial frame $S$, observers $A$ and $B$ will disagree on the time of the event P, as $A$ and $B$ are located at different positions and their clocks are synchronized by rule (2). Hence, we have that $A$ measures the spacetime coordinate of $P$ to be $ (t, x, y, z) = (t_A, x_P,y_P, z_P)$, while observer $B$ measures the spacetime coordinate of $P$ to be $ (t, x, y, z) = (t_B, x_P,y_P, z_P)$. Here, $t_A \neq t_B$.
Now we consider the $S'$ frame. Let observer $A'$ be located at $(x',y',z') = (x_A',y_A',z_A')$. Here, $x_A = x_A', \; y_A = y_A', \; z_A = z_A'$. Let observer $B'$ be located at $(x',y',z') = (x_B',y_B',z_B')$. Here, $x_B = x_B', \; y_B = y_B', \; z_B = z_B'$. In other words, $A$ and $A'$ and $B$ and $B'$ are located at the same position relative to their origins, (i.e., they "correspond").
Now surely, the Lorentz transformations would relate the measurements of the spacetime coordinates of the event $P$ made by $A$ and $A'$. They would also relate the measurements made $B$ and $B'$. However, those same transformations could not relate the measurements made by $A$ and $B'$ measurements or the measurements made by $B$ and $A'$ right? This is what I mean when I say that the Lorentz transformation only applies to corresponding observers.