When the EM Field tensor, F is introduced, it is usually assumed it is a tensor. How could this be shown?
You can define $F$ in terms of $A$, or $A$ in terms of $F$. It's a matter of taste.
If you take $A$ as fundamental, and assume that $A$ is a tensor, then $F$ is automatically a tensor because it's a derivative of a tensor. The index notation is defined so that any set of symbols that obeys the grammar of the notation is guaranteed to be a tensor. (In GR, the derivative has to be a covariant derivative, not a partial derivative. I assume whatever book you're referring to either isn't considering curved spacetime or uses $\partial$ as a notation for the covariant derivative.)
If you take $F$ as fundamental, then I think it's a lot more obvious why it's a tensor. The typical way to define it is that the four-acceleration of a charged particle is given by $a_\mu=(q/m)F_{\mu\nu}v^\nu$, where $v$ is the velocity four-vector. Thus $F$ is a linear operator that takes a four-vector as input and gives a four-vector as output, and that's one way of defining what we mean by a rank-2 tensor.
(and is it possible without going into the lorentz transformations or is it only a tensor with respect to them)?
When we say something is a tensor, we normally mean that it transforms (or, according to taste, its components transform) according to the standard transformation rules under any smooth change of coordinates, not just Lorentz transformations. This applies to $F$. (It is possible to have things like psuedotensors or tensor densities that have slightly different transformation properties, but $F$ isn't one of those.)
In addition to this, is the significance that it is a tensor that it becomes true in all reference frames (and therefore accounts for SR) or is there some other significance to it?
Your wording seems a little off to me, but basically yes, the reason we like to deal with tensors is because any tensorial relation remains valid under any smooth change of coordinates, which includes a Lorentz transformation.