What is the difference between a scalar, vector, matrix and tensor in simple terms? Are vector fields and tensors related?
A matrix is an array of numbers used in linear algebra. Various classes of matrices might form groups and other fun mathematical structures, but a lone matrix is not much more than a mathematical object.
Scalars, vectors, and tensors, on the other hand, are geometric objects. What makes them important is that these are the geometric object we encounter in nature. "Scalar" and "Vector" are special names for Rank-$0$ and Rank-$1$ tensors, while "Tensor" may refer to Rank-$2$ or Rank-$n$...depending on context.
They can be classified by how the transform under rotations. Scalars have the property of being completely spherically symmetric: they look the same no matter how you rotate them. Vectors, with their magnitude and direction, are the simplest non-trivial thing that can be rotated nicely (under normal 3D rotations).
Tensors are more complicated: naively they transform like the dyadic product of 2 vectors, and that requires 2 application of a rotation, which gets messy for 2 reasons:
1) Rotations don't commute
2) Cartesian coordinates obscure much of their symmetries
Wigner, Clebsch and Gordon, and others have addresses this. In summary: a "natural form" rank-$n$ tensor is symmetric, and traceless, and has $2n+1$ degrees of freedom that "look" exactly like, and transform exactly like, the $2l+1$ spherical harmonics of degree $l=n$: $Y_l^m(\theta, \phi)$. I find this to be the best way to understand their geometric nature (esp. since it easy to find pictures of them, e.g. atomic orbitals).
The biggest point of confusion when 1st exposed to them is the confusion between the geometric object (which is an abstraction of "thing" encountered in nature), and their representations, which look like matrices (at least up to rank-$2$). The numerical representation of a geometric object is not the geometric object. The former is a coordinate dependent thing, while the geometric object itself exists independent of any coordinate system, and people do try to formulate the laws of physics in a coordinate-free manner.
Another point of confusion is the notion, from computer science, that "scalar" means "number". While a number can represent a scalar, a number cannot be rotated: it is not a geometric object.
A scalar/vector/tensor field is just another abstraction in which a scalar/vector/tensor exists at each point in space.
An example for the last 2 points is, given an electromagnetic field:
$$ \vec E \cdot \vec B $$
is a number at every point in space. It's a (pseudo)scalar field. I you rotate the coordinates, $\vec E$ and $\vec B$ change, but their dot product remains the same. But it is not a number: if I reflect the coordinates it changes sign (which is way I added "pseudo" to it). Numbers don't change signs. Scalar are not numbers.
That being said, there is also a complementary view of rank-$n$ tensors as multi-linear functions that map $n$-vectors to a real number in a coordinate independent manner. This view is useful in the development of coordinate-free laws of physics.