Do electromagnetic waves occupy varying amounts of space, or do they simply vary in magnitude? 
(source: nde-ed.org)
The above diagram shows an electromagnetic wave propogating in the $x$ direction, if the electric field is in the $y$ direction and the magnetic in the $z$ direction.
I was taught however that the strength of an electric field is given by the 'density' of field lines in a region, and in the above graphic it seems that the density is always the same, but that the area the electric field occupies changes. This implies that the strength of the electric field along the x axis never really changes, but it just spreads into the y axis.
But then I thought perhaps it is not the space that the electric field occupies that changes, but the magnitude of the electric field, with this being the only way to show it graphically. So is an electromagnetic field just a single beam on the x axis with varying magnitudes of E and B, or does it extend into the y and z directions?
 A: Your last paragraph is right. The vectors in that picture are not really taking up a certain amount of space; they are simply vectors that exist only a the very base of where they are drawn. Their units are units of electric and magnetic field strength, not of length. We simply draw them as vectors pointing through a certain amount of space because we don't have a way of representing the entire vector all at one point.
An electromagnetic field extends all throughout space. The ideal picture would show electric and magnetic field vectors all over the place, one pair of vectors at every single point in space. Again, we simply don't know how to draw the electric field at every single point all at the same time. But for a plane wave, you can imagine that at any value of y and z there is a wave doing exactly the same thing as the one shown. When trying to draw the entire electric field, we normally use field lines instead of drawing a whole lot of vectors. Both type of pictures - vectors and field lines - can help with your intuition.
A: 
So is an electromagnetic field just a single beam on the x axis with varying magnitudes of E and B, or does it extend into the y and z directions?

The EM field extends in all directions and for most intents and purposes is omnipresent. If it helps, you can imagine the electromagnetic field as a bundles of field lines, but it's not vital for this question.
An electromagnetic wave however, is a propagating disturbance in the electromagnetic field. If you imagine pressure waves in the air, you could draw a diagram relating spatial position to air pressure - the diagram shown is basically the same concept, but with electric and magnetic potentials instead of pressure.
