After some more googling and time spent watching different explanations, I think I have got a better understanding.
This one in 2D shows the electric field propagating to the right. Its distance and time on the x-axis. It also shows that the electric field, mostly are not sine waves (as they typically are visualized). The magnetic field is perpendicular to the electric field - but I think it is a bit strange in this illustration, since the electric field probable is a "plane" and not a "thin line".
It is explained here that the electric field can have arbitrary shape, it all depends on how it was created. That the electric field is like a "sine wave" is said to be a misconception in that video.
From No, Changing Electric Fields DON'T Cause Magnetic Fields; The Real Origin of Electromagnetic Waves.
That explanation for the shape of the electric field makes sense to me. Especially since an electric field is radiated anytime we e.g. power on a circuit board, as explained in Veritasium - How electricity actually works
But this was only in 2D, and it does not really explain the magnetic field, other than that it is perpendicular to the electric field.
But this picture from wikipedia says little more about the fields in 3D, when radiated from an antenna.
This is actually not so different from how the fields are organized around a wire carrying electric current. As in this picture: 
.