If I'm sitting on a chair I'm exerting a force on it but my body isn't even moving so it doesn't even have a velocity. Then how can it even have a change of velocity i.e, acceleration?
Force is not equal to mass times acceleration.
Only net force is. This is a very important distinction. It might be helpful to always write Newton's 2nd law as:
rather than just $F=ma$.
When sitting in your chair, as you describe, you are pressed down by your weight. At the same time the chair pushes up with a normal force. Since they balance each other out perfectly, the acceleration is zero, $a=0$. This scenario is basically what we call newton's 1st law.
This is how wikipedia defines 'force'
In physics, a force is any influence that, when unopposed, will change the motion of an object. A force can cause an object with mass to change its velocity (which includes to begin moving from a state of rest), i.e., to accelerate.
Now in your problem, there are two forces acting on the you: weight and normal force. You are sitting still because weight is opposed by normal force. If there was only weight, you will accelerate downwards. If there is only a force equal to normal force exerted on you in the same manner, you will accelerate upwards.
Newton's second law states that the rate of change of momentum of a body over time is directly proportional to the force applied, and occurs in the same direction as the applied force.
As above said, force in the above context means an unopposed force. If there are more than one force, the above context refers to the resultant force. Resultant force is the unopposed force in that case. Since the net force acting on you in this example is zero, according to the mathematical representation of Newton's second law, $F=ma$, acceleration is zero.
You can think of the chair in the same way. I'll leave it to you to think.