Why do all the atoms of a radioactive substance not decay at the same time? Why does the substance decay at a rate which is proportional to the amount of the substance at that moment?
As all atoms are in hurry to become a stable atom and as their decay do not depend on any external things (like pressure, decaying of neighbouring atom), they all should decay in a moment into a stable atoms and the whole substance should become an stable substance.   
 A: All the atoms have the same chance to decay at any given moment.
If you have more of them at the same place, you will simply have bigger chance of them decaying.
It's like dice, you have a 1 in 6 chance of getting a 6.
If you have 100 dice, you will have 100 times more chances of getting a 6. Thus you'll have more 6-es.
Unless you cheat ;)
A: The decay phenomenon is a purely quantum mechanical property. This problem is equivalent to a particle in a finite potential well, and a lower potential state that is available outside the well. Classically if the energy of the particle in the well is lower than the potential barrier - it will never get to the lower state. By quantum mechanics, the particle can tunnel through the barrier to the lower state, but its chance to accomplish this is very very low (in actual situations). Also, its probability to tunnel is independent of the state of other particles and the previous attempts of the particle to tunnel don't change its probability to accomplish the next tunneling (no memory). Now, using some mathematical tools from the probability theory you can prove that the probability of each particle to tunnel has an exponential probability density, and so  the collective behaviour of many particles (which you asked about) follows that.
A: It's basically got to do with the fact that nuclear decay is a quantum mechanical process, and quantum mechanical processes are not deterministic in the traditional sense, i.e. given a set of conditions you can't predict exactly what will happen in a particular process, only the probability of something occurring. In this case, nuclear decay occurs through quantum mechanical tunneling, a process wherein particles will spontaneously "appear" on the other side of an apparently impenetrable potential barrier (in this case created by the nuclear force) because of the uncertainty in their position. There is an associated probability with this tunnelling process, and that probability determines the decay rate of nuclei. In a macroscopic "chunk" of matter, the nuclei appear to decay at a steady rate, but each individual particle has a large range of times it might decay at.
A: Speaking loosely, each individual atom has a desire to become stable, but that translates into a probability of decaying. This means, since there are billions and billions of atoms in a macroscopically significat chunk of material, that there are always going to be unlikely holdouts, and these holdouts are responsible for radiation that  after the initial instant of creation. 
If you think of time as a clock ticking and you have a lump of radioactive material where the atoms have an x% chance of decaying each tick, then every clock tick you go through the remaining atoms and ask them, "Are you decaying this time?" On average x% of them will say "yes."
There is also an observation bias in our general intuition. Most very radioactive elements have already decayed, which means that when we think of "radioactivity," we think of relatively inactive elements like cesium or uranium. These have half-lives measured in hundreds of millions or billions of years. Their atoms don't feel a great desire to become stable, compared to some of the very unstable elements people have created. Those elements' half-lives are measured in fractions of a second, and in that case, your intuition of "sudden spike of activity, then gone" is correct. In fact, if you were to regard a lump of uranium over a timescale many times its half-life (so, hundreds of billions of years) you would observe the same phenomenon: a huge spike or activity, then it's all but gone. However, compared to that timescale, our lives are instantaneous, and we're still living through the "instant burst" of activity following the creation of our solar system's uranium in that long-ago supernova.

Mathematically, (rewrite pending)
A: Aside from the obvious answer of randomness in probability distribution, each atom's decay event does depend on its overall energy, which cannot be measured individually by current technology. For example it may be contained in a gas where the density in one area of the gas is slightly higher than other areas and may be undergoing more "collisions" (interactions) and so its ground state differs from other atoms in the gas. Statistical mechanics cannot answer this question but new ideas in quantum entropy could, someday...
