This is a fairly common point of confusion. The arrows you see in that diagram express the magnitude and direction of the electric and magnetic fields on the $\hat x$ axis.
When you are introduced to vectors for the first time, they are usually represented as arrows which extend from one point to another. The vector $\langle 2, 3\rangle$ is depicted as an arrow which starts on the point $(0,0)$ and ends on the point $(2,3)$.
This is nice and intuitive, but it implies that the vector extends across physical space, which it emphatically does not. Vectors exist at isolated, individual points.
Here is a "slice" of the electric field of an electromagnetic plane wave defined over the plane $z=0$, rather than just along the $\hat x$ axis:
![enter image description here](https://i.sstatic.net/oElIX.png)
Now imagine identical copies of this plane stacked along the $\hat z$ axis, and you have a picture of what the electric field from a plane wave looks like. A faithful visual depiction would be far too cluttered to be useful, but this is the general idea.
For completeness, here's a slice of the field in the plane $y=0$:
![enter image description here](https://i.sstatic.net/8P7GY.png)