Why is $R\cos{a} = mg$ in circular motion compared and not $R = mg\cos{a}$? 
Normally, if an object of mass $m$ is inclined to the horizontal at an angle $b$, we set the reaction force of the object on the inclined plane as $R = mg\cos{b}$ (if we resolve the force of gravity so the line of action coming out of the plane is perpendicular to it).
However in circular motion*. it's assumed that $R\cos{b} = mg$. In the example above, one would have to do this in order to arrive to the correct answer, instead of $R = mg\cos{b}$. Using $R = mg\cos{b}$ seems natural enough, as I am resolving vertically, however, both equations would produce two different values for $R$. Why is this? 
To show what I mean:
If we set the reaction force in this question as $mg\cos{a}$, then the centripetal force will be $mg\cos{b}\cos(\pi/2-b) = mg\cos{b}\sin{b} = \frac{1}{2}mg\sin(2b)$ 
Whereas If we use $R\cos{b} = mg$, $R = mg\sec{b}$ and the centripetal force will be $mg\sec{b}\sin{b} = mg\tan{b}$. This will end up with two different values for the radius of the circular motion, and hence two different final answers.
*In the circular motion questions I've seen in my mechanics module
 A: Never, ever, just blindly memorize formulae.  
What you need to do is draw a free-body diagram of your particle, which will have an angled normal force, and a downward gravitational force, and you know that the net acceleration is inward with magnitude $v^{2}/r$.  You can either rotate your reference frame so that the normal force is upward, and the gravitational force is angled, or work out the two equations, eliminating the normal force.  
Either way, you'll arrive at an answer.  But the text of the question presupposes that you can just memorize a formula for a situation.  Never do this, look at a situation, and work out the answer.  You will end up wrong as often as you don't if you try and solve problems the way you seem to be -- because all it takes to be wrong is someone labeling an angle in a funny way, or using a slightly different convention.
A: @JerrySchirmer's advice is generally good, and worth heeding.  If you actually construct the free-body diagrams for a particle on an inclined plane and your particle on a cone, you will note the following important difference:


*

*A particle at rest on an incline (or sliding down an incline) has an acceleration vector that is parallel to the surface.  

*Your particle moving around the inside of a cone, on the other hand, does not have an acceleration vector parallel to the surface:  it is accelerating horizontally towards the center of the circle instead.  


In both cases, you can then use the fact that the particle is not accelerating in the "other" direction (perpendicular to the plane for the incline;  vertically for the cone) to write down a relation between the particle's weight and the reaction force.  But these respective equations deal with the components of those forces in different directions, and so they turn out differently from each other.
A: First, let's look at the difference between two scenarios:
Case 1: A box is standing still on an incline.
In this case, the type of motion we are trying to oppose is the box sliding down the incline, i.e. the effect of gravity force on the box. Don't split up the weight into parallel and perpendicular components yet. Draw the normal force and the friction force. Then draw their sum. You can see that their sum is vertical, opposite the direction of the gravity force on the box. If this sum is sufficiently large, it will fully counter the gravity force in the box and prevent it from sliding up or down.
Case 2: A car is moving around a banked road/race track, with no friction (coefficient of friction = 0).
In this case, the types of motion we are trying to oppose is:

*

*The car sliding down the incline, due to the effect of gravity force on the car.

*The car skidding out of its circular path and off the road.

Again, draw the weight/gravity force on the car without separating it into its components. A vertical force (or vertical sum of forces) in the opposite direction needs to counter this force. What's touching the car (on the outside)? Just the road under it! And there's no friction, so the vertical force can only be the vertical (y) component of the normal force/the force supplied by the road on the car. But we still need a horizontal centripetal force to keep the car in a circular path! Again, the only thing that can supply this horizontal force is the normal force/the road's force on the car. So the normal force must have a horizontal component that provides the necessary centripetal force.
Hopefully, you can see the difference between these two scenarios.
Now let's derive the respective formulas for each of these scenarios.
Case 1:
Draw the vector representing the weight of the box. Split this vector into its parallel and perpendicular components. The normal force counters the perpendicular component, which equals mg * cos(theta), where theta is the angle of the incline. The friction counters the parallel component, which equals mg * sin(theta).
Case 2:
Draw the vector representing the weight of the car (not its components). In this case, we have no friction, so all of the weight of the car (not just the perpendicular component) must be countered by the vertical component of the normal force alone. Thus F_n,y = mg. Using simple geometry, we can find that the angle between the net normal force (by definition, perpendicular to the road) and its vertical component is equal to theta, the angle of the incline. Thus, F_n = mg/cos(theta).
I hope this helps! I was stuck on this too for a while, and I decided I'd jot down the thinking process I went through to clear things up for myself. :)
