Why is Newton's first law necessary? Newton's second law says $F=ma$.
Now if we put $F=0$ we get $a=0$ which is Newton's first law.
So why do we need Newton's first law ? 
Before asking I did some searching and I got this: Newtons first law is necessary to define inertial reference frame on which the second law can be applied.
But why can't we just use Newton's second law to define an inertial frame? So if $F=0$ but $a$ is not equal to 0 (or vice versa), the frame is non-inertial.
One can say (can one?) we cannot apply the second law to define a reference frame because it is only applicable to inertial frames. Thus unless we know in advance that a frame is inertial, we can't apply the second law.
But then why this is not the problem for the first law?
We don't need to know it in advance about the frame of reference to apply the first law. Because we take the first law as definition of an inertial reference frame.
Similarly if we take the second law as the definition of an inertial frame, it should not be necessary to know whether the frame is inertial or not to apply the second law (to check that the frame is inertial).
 A: Newton's first law is necessary, because it does something.  Let's look at what the laws do.
Newton's third law constrains what force laws you consider (effectively you only use/consider force laws that conserve momentum).
Newton's second law turns these force laws into predictions about motion, thus allowing the force laws to be tested, not just eliminated for violating conservation of momentum. This works because he postulates that we can test force laws by using calculus and then looking at the prediction from solutions to second order differential equations.
Newton's first law then excludes certain solutions that the second law allowed. I'm not saying that historically Newton knew this, but it is possible (see Nonuniqueness in the solutions of Newton’s equation of motion by Abhishek Dhar Am. J. Phys. 61, 58 (1993); http://dx.doi.org/10.1119/1.17411 ) to have solutions to F=ma that violate Newton's first law. So adding the first law says to throw out those solutions.
Since you said F=0 implied a=0, let me point out that yes that is true, but Newton's first law says more than a=0 it says that it stays at rest if at rest and has the same constant motion if in constant motion.  The second law does tell us that F=0 implies a=0, but that does not mean that the velocity is constant, merely that the acceleration is zero, but if you have a nonzero jerk, then the acceleration can change. Jumping from a pointwise zero acceleration to a constant velocity is just like a student analyzing projectile motion, noting that the velocity is zero at the top and then assuming the projectile stays there forever (student thinks once the velocity is zero for an instant, that therefore the position stays constant forever after). The student ignored the possibility of a nonzero acceleration.  To jump from a zero acceleration to the velocity staying constant forever after is to simply ignore the possibility of a nonzero jerk.  It is exactly as big an error (to just assume that without a law or principle).  A body can experience no force at an instant (and hence no acceleration) and have no velocity at that instant and yet start moving again (if it had a continuous and nonzero jerk at that instant it would have to).  So Newton's first law has content, it excludes those motions.  And in fact it sometimes forces the jerk to be discontinuous.
In summary: the third law constrains the forces to consider, the second makes predictions so you can test the force laws, and the first constrains the (too many?) solutions that the second law allows. They all have a purpose, they all do something.
A: I have to say I found Newton's laws very uneasy to understand when I was in high school. I had a lot of questions similar to the OP's. I remember I asked my high school teacher exactly the same question in the OP's post. I was also confused by the meaning of mass, whether the second law is a law or a definition, etc.. My high school teacher couldn't answer my questions very well. To him, mass is just something measured by a balance. Newton's second law is a law in the sense that when you are given the force $F$, given the mass $m$, you use the law to obtain $a=F/m$ and then solve the motion.
I think it's not easy to understand Newton's laws in the ways and order they were presented by Newton, which is due to historical purpose may be. For example, I think Newton stated Newton1 as the first law because at his time most people believe in Aristotelianism. So he wanted to put his first law at the very beginning to emphasise that Aristotle was wrong.
I think I had a better understanding of Newton's laws until I read Feynmann's lectures. In my opinion, the best way to understand Newton's laws is in the order 2 --> 3 -->1. If I will become a high school teacher one day, I will teach my students in the following way.
First, we have the second law $F=ma$. So here we have two new things, $F$ and $m$. I will explain to my students what inertial mass is first.
I will tell them it is observed that when different objects are put under the same situation, e.g., being pulled by the same spring with the same extension, their accelerations are in general different. Some objects seem to be more reluctant towards acceleration than others. However, it is found that the $\textit{acceleration ratio}$ of two objects is always the same. Moreover, it is observed that this acceleration ratio is transitive, meaning that if the acceleration ration of object $A$ and $B$ is $m_{AB}$, the acceleration ratio of object $B$ and $C$ is $m_{BC}$, then the acceleration ratio of $A$ and $C$ will be $m_{AC}=m_{AB}\times m_{BC}$. The above then implies one can taken a standard mass call $1$ kg and then define the mass of all other objects by the acceleration ratio.
Now after defining $m$, I will simply take $F=ma$ to be the definition of force.
Then Newton's third law states that for any force, there is a reaction force. Or in other words, whenever you see something accelerating in one direction, somewhere else in the universe, you must be able to find another thing accelerating in the opposite direction. Forces with reactions are called real forces and forces without reaction are called pseudo forces.
Now, it's not difficult to find examples that Newton's third law is wrong. In other words, the observation of pseudo forces. For instant, when you are inside a train just leaving the platform, you see the people on the platform accelerating in one direction. You can define force according to $F=ma$ but you are not going to find the reactions. For someone on the train, the people on the platform are under no (real) forces, but are accelerating.
So Newton's third law is clearly wrong for some observers. Those observers who see pseudo forces are called non-inertial observers. For observers to which very force has a reaction are called inertial observers.
Then finally we come to Newton's first law, which then can be interpreted as a postulate of the existence of inertial observers. For inertial observers, when there is no (real) force, there is no acceleration.
A: 
Newton's second law says $F = ma$. Now if we put $F = 0$ we get $a = 0$ which is Newton's first law. So why do we need Newton's first law?

I don't think this is obvious from Newton's statement of the Second Law.  In his Principia Mathematica, Newton says that a force causes an acceleration.  Without the first law, this doesn't necessarily imply that zero force means zero acceleration.  One could conceive of other things that also cause acceleration. 
A modern person might be concerned about non-inertial reference frames.  Someone from Newton's time would probably be more concerned about Aristotelian ideas of objects seeking their own level.  But in either case, its necessary to emphasize that forces not only cause acceleration, but that they are the only things that do so (or in the modern formulation, that there exists a frame in which they are the only things that do so).  
A: Newton's first law postulates that there is (at least) one inertial reference frame for every object, in which said object will continue in uniform motion unless acted upon by a force.
Newton second law states that, within the inertial reference frame for any object, $F = ma$.
Without the first law to assert that there is indeed a frame in which $F=0$ implies $a=0$, the second law is vacuous.
