In the image included in your question, one should not view the arrows as literally extending through space or having a spatial length associated to them. The length of these arrows is the magnitude of the field, and they should each be thought of as living inside of their own vector space which is "internal" or "attached" to a given point in space. Your last statement is essentially correct, there is no "lateral extension".
As a very simple analogy, imagine a particle moving through space along some trajectory with coordinates $\gamma(t) = \{x(t),y(t),z(t)\}$. The trajectory traces out of a one-dimensional curve through space. At every point on this curve, we can talk about the velocity vector, $\mathbf{v}(t) = \dot{x}(t)\hat{\mathbf{x}}+\dot{y}(t)\hat{\mathbf{y}}+\dot{z}(t)\hat{\mathbf{z}}$. We often draw a picture of the curve and its associated velocity vector as if the velocity vector had a length in space, for example like this:
Does the velocity vector literally represent something with a spatial extent? Of course not. We should understand the velocity vector as living in the tangent space $T_{\gamma(t)}E$ at the point $\gamma(t)$, where $E$ is the spatial manifold, usually taken to be Euclidean space. There is a separate tangent vector space for every point in space, and these are the spaces where each of the velocity vectors live for each point of the trajectory. Note that the direction that the arrows point has a meaning with respect to space, but their length only tells you the magnitude of the velocity (i.e. the speed). We don't think of the velocity vectors as representing spatial displacement vectors (they can't since a velocity vector has magnitude measured in units of length/time while a displacement vector has a magnitude measured in units of length).
In fact, the electromagnetic field should properly be understood as living in the space of 2-forms at each point of space, $\bigwedge^2 T_pM$ for each point $p$ in $M$, where $M$ is 4-dimensional Minkowski space.
The basic idea is that the "fields" we talk about in physics (including quantum fields) take a value at every point in space(time), and the value they take lives in an "internal space" at each point. This notion is formalized in the idea of a fiber bundle, where the fibers are the "internal spaces" connected to each point of space.
Addendum
Remember that the field takes independent values at every point in space. I made a little animation of a plane wave to show this. Note that in the image below each little arrow represents the local field strength and direction, but these arrows each live in the internal space. I can only show a finite number of arrows, but there are more arrows filling all of space in the gaps. Try focusing on just a single arrow.