Cross products are inherently useful when describing rotations.
First, let's look at two different ways of describing rotations in $\mathbb{R}^{3}$.
The first way to do this is to give the axis of rotation, which is given by a line, $L$, in $\mathbb{R}^{3}$, and a magnitude (representing the angle), which is given by a number, $\theta$, in $\mathbb{R}$. If I pick a vector of length $\theta$, along my line $L$, I get a vector, say $x \in \mathbb{R}^{3}$.
Another good way to do this, is to give the plane in which I'm rotating, which I can represent by two perpendicular lines in $\mathbb{R}^{3}$ and a magnitude (representing the angle), which is again a number in $\mathbb{R}$. I encode these things by picking two vectors $v,w \in \mathbb{R}^{3}$, and say that the magnitude is encoded by the product of the lengths $\|v \| \|w\|$.
This means that lots of different pairs of $v,w \in \mathbb{R}^{3}$ give the same rotation, but that's ok.
(I can even allow more different pairs, by not assuming that $v$ and $w$ are perpendicular, but then I have to replace their product by the area of the parallelograms spanned by them.)
Now, the cross product gives us a way to translate between these different ways of encoding rotations.
To be precise, if $x \in \mathbb{R}^{3}$ and the pair $v,w \in \mathbb{R}^{3}$ describe the same rotation, then $x = v \times w$.
(The fact that lots of different pairs $v,w \in \mathbb{R}^{3}$ describe the same rotation means that $x$ can be written as the cross product in lots of different ways, i.e., there are lots of $v',w' \in \mathbb{R}^{3}$ such that $v' \times w' = v \times w = x$.)
Now, why this turns up in physics doesn't have such a clear cut answer, except that both these two different ways of representing rotations have their uses.
For instance, in your example talking about a charge moving in an electric field, I would say that this is just a fact of nature that was established experimentally.
An interesting aside is that rotations can be composed, i.e., given two rotations I can first do one and then the other to get a third rotation.
It might be interesting to try to figure out how this works in either of the pictures I have given above.