Why does a ceiling fan start up slowly? I think it probably has something to do with the capacitor inside but I don't get it why doesn't it just Start spinning instantly when we push the button why does it slowly start to spin and gradually gets faster?
 A: It's a bit complicated (Wikipedia). Induction motors work in sync with the AC frequency but have no torque at 0 RPM so they need some arrangement to get them started.

A: When the fan starts spinning, each blade starts from rest. Newton's first law of motion states that unless a force is applied to it, the [velocity][1] of a body does not change. That property is inertia. To speed up, the blades must accelerate. Newton's second law of motion states that the force necessary for an acceleration of a body is proportional to and parallel to the acceleration, and the ratio of the force magnitude to the acceleration magnitude is the mass of the body. Newton's laws of motion describe point particles in linear motion, and the blades are extended bodies in rotational motion. We can understand the blades' motion from Newton's laws, nonetheless.
A (point) particle is a body where all mass is concentrated at one point. It is an idealization, but we can understand real bodies using this concept. An extended body is a body where the mass is spread over a region of space. We can think of the multiple blades of the fan, as several extended bodies, each of which is made up of many particles. When the fan spins, each of the particles moves in a circle.
Consider a single particle. It starts at rest. According to Newton's second law of motion, it will accelerate in the direction of a force applied to it by an amount proportional to the magnitude of the force. The motor applies a force to the particle through the fan blade. If the particle were not attached to the fan blade, it would accelerate in a straight line. The crucial point is that the magnitude of the acceleration is proportional to the force applied. For the fan to reach full speed instaneously, the particle would need to accelerate to full speed instaneously. That is an infinite magnitude acceleration which requires an infinite magnitude force.
The particle is part of the blade, so the blade exerts a force that prevents the particle from moving away from the center of the fan. The result is that the particle accelerates through a very small arc in space in a very small span of time. Then the motor exerts a force on the particle in the new position in a slightly different direction through the blade, and the process repeats. So the particle accelerates in an arc that eventually becomes a circle, and it continues accelerating until it reaches the final speed.
The same process occurs with all of the particles that make up all the blades until all the particles reach their full speed circular motions. Collectively, this process is the process of the fan accelerating in its rotational motion to full speed. Others have explained this more concisely in more technical language. So, the ceiling fan starts slowly because the motor cannot exert an infinite force.
[1]: Velocity is the rate and direction that position changes.
A: The fan motor provides a torque $\tau$ which has to accelerate $\alpha$ the fan blades whose moment of inertia is $I$:
$$\tau=I\alpha$$
Given how long it takes for the fan blades to stop the frictional torques must be fairly low and so the torque applied by the motor to keep them going must also be low.  With the relatively small torque rating, even if the motor applied maximum torque when starting from rest, it would still take a noticeable period of time for the blades to reach their final rotational speed.  
The capacitors are there to make sure the motor rotates in the right direction when switched on and/or to allow the speed of the motor and hence the blades to be controlled.
A: A much simpler way of thinking about this is to consider energy.  When the fan is spinning it has quite a lot of kinetic energy (try to stop it by putting your finger in the way to confirm this (don't actually do this!)).  That kinetic energy goes as the square of the rotation rate, in fact.
So as the fan starts, the motor needs to add energy to it.  It does this by taking energy from the electricity supply.  Well, the amount of energy it takes from the supply per second is the power of the motor.  If it is going to start the fan really quickly it needs a great deal of power during the period the fan is spinning up.  That means that it would need both enormous cables running to it, and the motor itself would be huge.  Once the fan is spinning it needs much less power, so all of this is only needed during the spin-up process.
Economics, as well as the desire not to have enormous motors and cables attached to the ceiling, almost certainly requiring special support beams, possible liquid-cooling arrangements and so on, mean that rather small motors get used, and people live with a few seconds of spin-up.

Of course, real ceiling-fan enthusiasts get around this by using a small solid-fuelled rocket system to do the initial spin-up.  Suitably specified, these can produce many megawatts for a small fraction of a second, and also solve the problem of the torque tearing the house off its foundations that the very high torque motors used previously have.  Such systems can spin the fan up in tiny fractions of a second, and the achievable rotation rate is really limited only when the tips of the fan blades go supersonic, which tends to destroy the fan.  It is best not to be in the room during the spin-up: I believe most people retreat to an underground bunker at least a quarter of a mile away.
A: You might be thinking in comparison to a desk or handheld electric fan.

As mentioned by @Farcher, $\tau = I\alpha$. $I$, the moment of inertia of a spinning body around a particular axis of rotation, is calculated as follows:
$$I = \iiint\rho(x,y,z)||r||^2\ dV$$
Or with uniform density,
$$I = \rho\iiint||r||^2\ dV$$
From this formula, you can see that the moment of inertia of a rotating object increases as its mass is distributed further from the axis of rotation.
I'm going to spare the calculus and look up some formulas in a table so we can verify this easily:
Rod of length $L$ and mass $m$, rotating about its center: $I = \frac{mL^2}{12}$
Rod of length $L$ and mass $m$, rotating about one end: $I = \frac{mL^2}{3}$
Thin circular hoop of radius $r$ and mass $m$: $I = mr^2$
Thin, solid disk of radius $r$ and mass $m$: $I = \frac{mr^2}{2}$
Notice in all of these, the $r$ or $L$ is squared. So since the ceiling fan has longer blades on it than a desk fan does, it'll have a much higher moment, which is responsible for your observation that the $\alpha$ (angular acceleration) is much lower.
