It actually is possible(*) to find a purely algebraic way to analyze such problems mathematically, without using your hands or equivalent devices. The method is actually quite closely related to ggcg's answer, though it will look quite different. And it goes beyond that by actually dispensing with the cross product entirely, and replacing it with what we call the "wedge" or "exterior" product. Rather than taking two vectors and getting one other vector out of it, as in $\vec{a} \times \vec{b}$, when you take the wedge product you get a plane out: the wedge product $\vec{a} \wedge \vec{b}$ represents the plane spanned by $\vec{a}$ and $\vec{b}$, which is orthogonal to the vector $\vec{a} \times \vec{b}$. Moreover, there is also a "sense" of this plane, which substitutes for the right-hand rule, and is related to the order in which you take the product: $\vec{b} \wedge \vec{a}$ represents the same exact plane, except with the opposite "sense", just as $\vec{b} \times \vec{a}$ represents the same vector in the opposite direction. You can rewrite expressions for things like the Lorentz force law and rotations (and everything else in physics that uses a cross product) with the wedge product, and it all just works out beautifully. But in this case, rather than using the right-hand rule, you just need to decide on an order for your basis vectors: do you order them as $(\vec{x}, \vec{y}, \vec{z})$, or as $(\vec{x}, \vec{z}, \vec{y})$, for example? Making a choice for that ordering allows you to completely specify everything you need, expand in the basis, and compute the wedge product correctly without ever referring to handedness.(**)
This approach is just one small part of something called "geometric algebra". One nice feature of this approach is that it actually generalizes to other dimensions. In two dimensions, you get a nicer understanding of complex algebra. In four dimensions, you can use the same exact techniques to do special relativity more easily. It's actually just a coincidence that the cross product even works in three dimensions; if you want something that will work in three dimensions and any other dimension, you actually have to go to the wedge product.
Geometric algebra is actually a really great pedagogical approach, and we would all be better off if everyone would just use it, and ditch cross products forever. Unfortunately, that's not going to happen while you're a student. So while I encourage you to learn geometric algebra, and I would bet that you'd really get better at physics if you do, remember that you'll still probably be taught and asked to use cross products.
(*) I just need to dispense some practical advice here. It sounds like you're a relatively new (and probably talented) student of physics. Physicists (including teachers) are all very familiar with the various problems with the right-hand rule. And it is unfortunate. I wish that it weren't so, but as a purely practical matter, if you stick with physics you'll need to keep interacting with the cross product for at least another couple years. So my advice is to stick with it, and become really good at using it correctly. It's not that hard, and maybe it'll exercise your brain in ways that come in handy.
(**) It's still true that you might eventually need to relate your particular choice of directions to someone else's idea of the "correct" orientation. Basically, you would need to make your choice of ordering consistent with their choice, which would probably be based on the right-hand rule. However, unless a question specifically asks for that orientation, you could very well use a left-handed orientation and still get correct answers (e.g., force going in or out of the page, etc.) using the wedge product.