According to right hand rule, If I put my thumb in the direction of the current flow and encircle myt other fingers, the direction of those finger will refer to the direction of the magnetic field. But why does this work?
I mean, why is the magnetic field created in that direcrion?
It's an arbitrary choice, because the direction of $\vec B$ is not actually an observable.
Whenever you compute observables in electromagnetism --- for instance, whether two parallel currents are attracted or repelled, or whether two skewed currents experience an aligning torque or an anti-aligning torque --- you always find yourself using the right-hand rule an even number of times. For instance, you use the right-hand rule to find the direction of $\vec B$, then use the right-hand rule again to find the direction of $\vec v \times \vec B$. If you were to consistently use your left hand in every circumstance, you'd disagree with other people about the direction of $\vec B$, but you'd predict all of the same dynamics.
This property of electromagnetism, where it doesn't matter whether you use your right or left hand to compute the direction of a vector product, is known as "conservation of parity." While electromagnetism doesn't change under a parity transformation (which transforms your right hand into a left hand), that's not a generally true statement about the world: in the weak nuclear interaction, there are different rules for interacting particles with spin, depending on whether their spin axis is parallel to their momentum (i.e. "north pole forward") or antiparallel ("south pole forward").
It works because we use the same right hand rule to determine the force the magnetic field exerts on a current. Yes, the right hand rule is arbitrary - a left hand rule would have worked equally well, other than, maybe, forcing the majority of students to use a hand they're less dexterous with to do hand dances during exams. And it is always the case that these sort of right/left hand rules occur in pairs that make the observable result not arbitrary.
I'd try to elaborate on what I'm saying right now, this is how I understood it.
First of all, have a look at vector cross products if you aren't so sure what's going on inside Biot-Savart rule and related stuff. Then you'll be presented again with the right hand rule. By saying right hand rule, I'm NOT talking about the right hand grip rule(which is identical to what this content elaborates on), but the rule discussing about three perpendicular vectors. Now why? Why does the right hand rule work? Let me tell you, it does so only in the realm/paradigm of physics we all have agreed upon. It's purely a definition. There is NO actual direction of the perpendicular vector obtained by vector cross product, its an arbitrary choice, but we the humans, have defined that it happens to be in a certain direction. But in reality there is no real direction(such as popping into/out of the plane) for that vector obtained. We use the right hand rule which will agree upon all the derivations and results obtained by those who came before. To be precise, those derivations and results also place their foundation upon these rules.
If this still seems absurd, think of the direction of the magnetic field. Have you ever seen one? Have you ever witnessed one? You may have witnessed, but when you say that you feel that magnetic field, it's actually a magnetic force that you're feeling. Will you be able to witness magnetic field alone, without a magnetic force? No! But we treat the magnetic field as a vector to facilitate our understandings. Magnetic field is something that makes magnetic force happen. There is no meaning to magnetic field without a magnetic force. It's purely virtual which we've used to describe everything better.
