Is any matrix a tensor in special relativity?

My question is inspired by the definition of the electromagnetic field tensor in Carroll's _Spacetime and Geometry book._ In equation (1.69), he defines a matrix which he says is the (0,2) electromagnetic field tensor.

In section 1.8, he states the Maxwell equations in tensor notation, and additionally states that in this form they manifestly transform as tensors. I assume that this is due to the tensorial nature of the electromagnetic field tensor; however, I don't understand why it is a tensor.

Would we have to prove that the electromagnetic field tensor transforms as a tensor by brute calculation? Alternatively, is there some clever and more obvious shorthand that you could use to come to the same realization?