If the distribution of dark matter is not uniform, why it is not attracted towards center of galaxies? The dark matter hypothesis was formed due to difference in theoretical and observed speed of the outer parts of galaxies. Therefore, there is more dark matter in the outer parts of the galaxy than in the center. But, if it interacts gravitationally, why it is not attracted to the center, remaining at the edge of the galaxy?
 A: The usual idea of how dark matter (DM) is distributed in a galaxy is that it is densest in the middle and the density decreases with a slope between $\propto r^{-1}$ and $r^{-3}$. Note that this means that most of the DM mass is further out, but this is mostly because most of the volume is further out.
I suspect your main confusion comes from statements like 'DM is needed in the outer parts of galaxies'. What this means is that the local fractional amount of DM (compared to stars and interstellar gas) is larger in the outer parts of galaxies - in the centre there's still a lot of DM in terms of density, but the density of stars is much higher still, so the relative impact of the DM on the galaxy dynamics is small. There are some galaxies, however, notably dwarfs (objects with stellar masses less than a billion solar masses or so), whose dynamics suggest that the fractional contribution of DM is large everywhere, including at the very centre. 
A: 
Therefore, there is more dark matter in the outer parts of the galaxy than in the center.

It was the galaxy rotation curves that brought up the need for dark matter:


But, if it interacts gravitationally, why it is not attracted to the center, remaining at the edge of the galaxy?

The popularized explanation in the caption of the figure "by adding a dark matter halo surrounding the galaxy" leads to the misunderstanding.
The simplest answer is that it is attracted to the center, that is why it is on a rotational curve. The statistics of different masses and angular momentum create the shape and the density as matter is gravitationally attracted. 
It does not mean there is no dark matter in the center of the galaxy shown in the plot, it just means that it is more diffuse and dominates at the tail while baryonic matter as denser and heavier dominates in the center. 
There are models that assign normal  baryonic matter to the tail of the spiral, considered not very probable, and models with new stable particles to be found in high energy accelerators when the energies go higher. It is an open research question what type of matter the tail is made of, but not why it is seen at the tail. It is a statistical matter of density and masses.
A: But, if it interacts gravitationally, why it is not attracted to the center, remaining at the edge of the galaxy?
I think the problem is that you're probably imagining that the dark matter is just sitting there, not moving, and so it seems strange that it hasn't long ago fallen into the center of the galaxy. But this is like asking why stars in the outer parts of galaxies haven't fallen into the center of the galaxy, or why planets and asteroids and comets in the outer Solar System haven't fallen into the Sun.
The answer is that anything interacting gravitationally will be moving on orbits. If the orbits are close to circular (like those of planets and asteroids in our Solar System, or some of the stars in the outer parts of galaxies), then they naturally stay away from the center. If the orbits are highly elliptical ("radial"), then they do travel in close to the center. But then they move back out again, and they spend most of their time in the outer parts of their orbits (like comets do in the Solar System). Simulations suggest that most dark matter particles should in fact be on rather elliptical orbits. So most of the dark matter spends most of its time at large distances from the center, even if individual particles sometimes travel close to the center.
