Is the electromagnetic field strength tensor a tensor? The electromagnetic field strength tensor is given by:
$$\Bbb{F}^{\mu \nu}=\begin{pmatrix} 0 & E_x/c, & E_y/c & E_z /c \\ -E_x/c & 0 & B_z & -B_y \\ -E_y /c & -B_z & 0 & B_x \\ -E_z/c & B_y & -B_x &0 \end{pmatrix}$$
I know that this is a Lorentz tensor meaning (in accordance with the definition of such a tensor) that:
$$\Bbb{F}^{\mu \nu}=\Lambda^\mu _\alpha \Lambda^\nu_\beta \Bbb{F}^{\alpha \beta}$$
for a Lorentz transform $\Lambda$. But is it actually a fully fledged tensor, such that for any coordinate transformation we have:
$$\Bbb{F}^{\mu \nu}= \frac{\partial x'^\mu }{\partial x^\alpha} \frac{\partial x'^\mu}{\partial x^\beta }\Bbb{F}^{\alpha\beta}$$
Either way can it be easily proved and if not why do we call it the 'field strength tensor'?
 A: Yes, it is.
First argument: Look at the Lorentz force law.
The electromagnetic field strength tensor is the one for which $$m\frac{d^2x^\mu}{d\tau^2}=q F^{\mu}_{\ \ \ \nu}\frac{d x^\nu}{d\tau}$$
so that $q F^{\mu}_{\ \ \ \nu}\frac{d x^\nu}{d\tau}$ must be a vector for all possible velocities you plug in. This tells you how the $\mu$ index transforms. We already know how $\frac{d}{d\tau}x^\nu$ transforms (it's also a vector), and so we can conclude how the lower $\nu$ index transforms.
This is nice and easy, because it relies on the fact that $\frac{d x^\nu}{d\tau}$ is a nice, geometric object - it is normalized tangent to a path in Minkowski space. It's a vector and it so it transforms as a vector. Likewise with $\frac{d^2 x^\mu}{d\tau^2}$. Those two transformation laws and the Lorentz force law force $F$ to transform like a tensor, period.
Second argument: It's defined that way.
I think the confusion here is that you have the mathematical formulation of the general coordinate invariance of tensors, but you don't have the physical formulation of electrodynamics in general coordinates. That would make the problem very difficult to think about!
Maxwell's equations are:
$$F^{\alpha\beta}_{\ \ \ \ \ \ ,\alpha} = \mu_{0} J^{\beta}$$
$$(\frac{1}{2}\epsilon^{\alpha\beta\gamma\delta}F_{\gamma\delta})_{,\alpha} = 0$$
where the subscript comma denotes differentiation with respect to that coordinate. i.e., $\phi_{,\alpha}=\frac{\partial}{\partial x^\alpha}\phi$. There is nothing going on here besides new notation - this is exactly what Maxwell did, and there is no difference!
Note also that $J^\beta$ is obviously a clear, true, physical vector. It points in the direction of real moving charges.
To work in general coordinates, even in flat spacetime, things get messy. There is a very sophisticated and difficult technique to dealing with the mess: Replace every comma "," (the derivative) with a semicolon ";" (the covariant derivative). OK, it's not that difficult. The semicolon hides all the extra terms which are added when you do a coordinate transformation. (What do I mean by this? The Laplacian of a scalar $\phi$ is equal to $\phi_{,\alpha\beta} g^{\alpha\beta}$, if $g$ is the three-dimensional (1,1,1) metric. If you do a coordinate transformation, the metric components change, and now the Laplacian is equal to $\phi_{\ ;\alpha\beta} g^{\alpha\beta}$. This hides all of the first derivative terms and what-not. Evidently THE Laplacian in arbitrary coordinates is $\phi_{\ ;\alpha\beta} g^{\alpha\beta}$, and $\phi_{,\alpha\beta} g^{\alpha\beta}$ is the Laplacian only in Cartesian coordinates.)
The theory of electrodynamics is now defined as the theory of a tensor field $F^{\mu \nu}$ together with the equations:
$$F^{\alpha\beta}_{\ \ \ \ \ \ ;\alpha} = \mu_{0} J^{\beta}$$
$$(\frac{1}{2}\epsilon^{\alpha\beta\gamma\delta}F_{\gamma\delta})_{;\alpha} = 0$$
This is now a true statement which is valid for all coordinate systems, even weird/bad/arbitrary ones: Cylindrical coordinates, spherical ones, parabolic coordinates, whatever you please!
There is no new physics in what I've said so far. This is just the differential geometry of weird coordinate systems, and Maxwell electrodynamics in weird coordinate systems. But the previous two equations hold just as well in curved spacetime, with no modifications, and that's nice.
A: It can be easily proved. But for almost every easy prove, there is some abstraction involved.
Luckily, I think I can offer some insight. Modern physics is very related to differential geometry and the main idea is that pure geometrical objects should have definitions which are independent of coordinates. Unfortunately, I don't have space and time to explain what is a proper geometrical object here, so the previous phrase will work as a definition. 
That said, if you can define the $F_{\alpha \beta}$ without ever mentioning coordinates, you will know automatically that it is independent of which coordinates you choose. 
The twist is that $F_{\alpha \beta}$ is not invariant as you pointed out, but
the quantity $F = F_{\alpha \beta} dx^{\alpha} \wedge dx^{\beta}$ is. So, if you could define $F$ before $F_{\alpha \beta}$, you would automatically know how this field strength tensor transforms from the requirement of $F$ being geometric. This is possible, but I cannot do it here. It is done, however, in every differential geometry book.
Let me explain a bit the notation.
$\bullet$ The quantity $F$ that I wrote is called 2-form;
$\bullet$ The symbol $\wedge$ is usually known as exterior product. It is there so that we can remember the antisymmetry:
$$ d x^{\alpha} \wedge d x^{\beta} = - d x^{\beta} \wedge d x^{\alpha}; $$
$\bullet$ The $dx^{\alpha}$ are known as basis $1$-forms. They are a basis for an abstract linear space usually known as cotangent space of a surface. This much you should not bother at the moment. The important thing is that you can sum and multiply them by scalars.
Under an arbitrary differentiable change of coordinates $x' \mapsto x$, the $1$-forms change to
$$ dx^{\alpha \, '} = \frac{\partial x^{\alpha \, '}}{\partial x^{\beta}} d x^{\beta}$$
and you can prove with the transformations you wrote (and I really leave this to you) that
$$ F'_{\mu \nu} (x') \, d x^{\mu \, '} \wedge d x^{\nu \, '} = 
F_{\alpha \beta} (x)\, d x^{\alpha} \wedge d x^{\beta} .$$
Thus, $F$ looks the same under any change of coordinates. It is therefore
independent of which coordinates you choose and geometrical quantity.
Here, I reversed engineered the logic just so you can see how it works.
The direct way to prove what you want
is to define first $F$ and then work out $F_{\alpha \beta}$. 
As a final comment, notice that two Maxwell's equations can be written in a manifestly invariant way under differentiable change of coordinates. They are known as Bianchi identities:
$$ \partial_{[\mu} F_{\nu \sigma]} = 0 \Leftrightarrow d F = 0. $$
The other two cannot be written in this way because they involve the metric, or more precisely, the Hodge dual. I should emphasize that diffeomorphic invariant things have geometrical significance and then you would know how their components transform. This is but one of many justifications of how it pays to have this level of abstraction involved.
A: Yes, because $F_{\mu\nu} \equiv \partial_\mu A_\nu - \partial_\nu A_\mu$. Hence, it is manifestly a rank-2 tensor since it is a sum of products of rank 1 tensors (vectors).
EDIT: This question is much more nuanced than I had originally thought. In order to actually show that it obeys general covariance, that is $F_{\mu\nu}$ is a tensor, we must generalize to Maxwell's equations in curved spacetime. That is to say, we must allow for accelerating frames of reference. Then, we know that $D_\nu F^{\mu\nu} = J^{\mu}$, where $D_\nu$ is the covariant derivative. Now, if $J^\mu$ is a vector, then it follows that the $\mu$ component of $F^{\mu\nu}$ also transforms like a vector (since we can pull coordinate transformations across $D_\nu$). By antisymmetry, this means that $F^{\mu\nu}$ transforms like a tensor.
Why must $J^\mu = (\rho, \mathbf{j})$ be a vector? To my understanding, this has to be argued on physical grounds, and is an assumption made outside the framework of Maxwell's equations. I could be wrong about this last statement, would appreciate an independent verification.
The moral of this story is that, yes the EM field strength should be a true tensor because we expect that, due to general relativity, the principle of general covariance should hold.
