Why do unstable nuclei form? Why do unstable nuclei form?  Is it that we simply find unstable nuclei in nature and understand what these nuclei do in order to become more stable?
I feel like textbooks gloss over this question when addressing radioactivity.
 A: Unstable nuclei form in many different ways. First of all, many are created in old stars, especially when they explode as supernovae. This is the case for all elements with atomic numbers higher than Iron, as they are not formed during the normal life of a star.
As a result, elements like $^{238}U$ with a half life of 4.5 billion years, were formed in supernovae and are older than the solar system. However $^{235}U$ has a half life of only 700 million years. Only about 1% of any $^{235}U$ formed before the solar system is still around. This problem is even worse with isotopes with even shorter half lives, e.g. $^{14}C$ which has a half life of only 5700 years.
Such isotopes are formed here on earth through various processes. For example, $^{14}C$ is created when particles (cosmic rays or solar particles) strike $^{14}N$ atoms 
A: "Why" questions in physics can be answered in two ways:
1) that is what we observe
2) we have a mathematical model that explains observations and details how unstable nuclei form. The model is validated by correct predictions, and we can explain how unstable nuclei can exist.
We are in the second mode: given that unstable nuclei exist, we can explain how and why they exist in that particular form using our models.
The models are quantum mechanical and also dependent on special relativity.
As with the atomic models, they predict energy levels for the nuclei, on how protons and neutrons can fill energy levels fulfilling quantum number conservation and the Pauli exclusion principle.
With the atomic model, the electrons are in shells around the nucleus, and instability appears through interactions, when electrons are in exited states and there exist lower energy levels to which the atom can relax by emitting a photon.
With the nuclear shell model

The evidence for a kind of shell structure and a limited number of allowed energy states suggests that a nucleon moves in some kind of effective potential well created by the forces of all the other nucleons. This leads to energy quantization in a manner similar to the square well and harmonic oscillator potentials. Since the details of the well determine the energies, much effort has gone into construction of potential wells for the modeling of the observed nuclear energy levels.

Instability will appear in decay products of long lived but unstable nuclei, or in interactions, as happened at nucleosynthesis time in the cosmological models.
It is a many body problem. Each nucleon in an effective potential is created by the presence of all the other nucleons, obeying all the quantum rules, including energy conservation and angular momentum conservation.
Solutions exist in the model for several energy states. As in all quantum models, a higher energy state will fall down to a lower energy state if all quantum number conservation rules allow it. The higher energy states have a probability (this is quantum mechanics and it is all about probability) to decay to the lower states, and that can be computed as a lifetime for the decay. The shell models of the nucleus can predict the decay lifetime and the decay products, because in general the shell model is a validated model. Thus if a lower energy level exists for the nuclei and conservation of quantum numbers is not violated, there will be a decay to a  final nucleus with a stable (long lifetime) energy level in the shell model.
So the mathematical answer is: because of quantum mechanics and conservation laws. It is worth reading the link provided.
A: There are a few different ways that unstable nuclei are produced:


*

*Nuclear fusion is quite a common way to produce unstable nuclei in nature. At high enough energies, stable nuclei can fuse together to create unstable ones. For example, one step of one of the usual hydrogen-burning sequences in stars combines a helium-3 nucleus and a helium-4 nucleus (both of which are stable) into a beryllium-7 nucleus, which is unstable, with a half-life of roughly 53 days. Stars generally use nuclear fusion to produce most elements from boron up to roughly iron in their lifetimes. It's also how we produce many of the heavy synthetic elements in the laboratory, when we collide ion beams with a fixed target.

*Neutron capture can also turn a stable nucleus into an unstable one. Since neutrons are uncharged, they are unaffected by the Coulomb repulsion of the protons of the nucleus and can incorporate themselves rather easily into even a stable nucleus at the right energy. Even common building materials like concrete and steel can become radioactive in the presence of enough neutron radiation at the right energy. Neutron capture can even induce nuclear fission, and in fact this is the mechanism by which nuclear fission reactors operate. Oftentimes these reactors are kickstarted using a "neutron gun" which injects neutrons of the right energy into the reactor core.
Neutron capture plays a prominent role in the creation of even heavier elements, in more extreme conditions like supernovae, neutron star mergers, and other cataclysmic events. In a stellar nucleosynthesis process such as the r process (short for "rapid neutron capture process"), seed nuclei capture neutrons to move to heavier and heavier masses. Those heavy isotopes are unstable, and beta decay toward stability. Most of those heavy nuclei have extremely-short half-lives, but some r-process nuclei are long-lived enough to be found on Earth.

*Decay of other unstable nuclei is a rather obvious one, but still needs to be included as it's a distinct process. Most of the nuclei we see on Earth with short half-lives are themselves the decay products of unstable nuclei with longer half-lives. For example, the radon gas that accumulates in basements is one of the decay products of uranium-238 that has been in the soil basically since the Earth was formed.

*Neutrino interactions are a tiny, but notable, contribution to nucleosynthesis. A high-energy neutrino has a small, but nonzero, probability of knocking a proton or neutron out of a nucleus. In supernovae, there are an absolutely staggering amount of neutrinos produced (obligatory xkcd what-if: https://what-if.xkcd.com/73/); since there are so many high-energy neutrinos flying around, there are actually a non-negligible number of neutrino-induced nuclear reactions that happen, and it's currently believed that neutrino-induced nucleosynthesis partly explains the observed abundances of some light odd-numbered nuclei like fluorine-19.


This is not necessarily an exhaustive list, but you'll notice that it contains both examples found in nature and examples produced in the laboratory.
A: There's no fundamental principle that makes unstable states unable to exist. It's just that by being unstable, they won't exist for a long time. For example, take a cone. You could sit the cone on a table with its base at the bottom, and that would be stable ("stable" here means that if there is a small perturbation, the object settles back to its original state). You could also sit the cone on a table with its tip at the bottom, and that would be unstable. The unstable state won't remain for long - the slightest wind will cause the cone to topple over - but in principle, you can do it.
The same goes for unstable nuclei. You can make unstable nuclei - and they are made, in stars for example, or particle accelerators. You don't expect them to last very long, and many indeed do not (although there are also unstable nuclei that last for millions of years), but you can still make them.
Why make them? In stars, they're simply a consequence of the other things that are going on. In particle accelerators, it's because we want to make them for whatever reason.
A: It seems that you're thinking that unstable nuclei must be formed by a special process that other nuclei are not. This is incorrect: they are formed by exactly the same processes as stable nuclei, and if you want to know what those are, your question is "how do nuclei form?" with no distinction with regard to stability. The basic answer to that is they are formed by either pressing together smaller nuclei or breaking apart larger ones, and keeping in mind that a single proton or a single neutron are still also nuclei - the smallest possible, just as a group containing 1 person can still be thought of as a "group" in a natural sense.
With regard to how unstable nuclei become stable, the answer is that they do so through decay, i.e. radioactivity.

Now if you're also wondering, "why are some nuclei unstable and others stable?", that's the cool part. You see, the atomic nucleus is basically a three-way tug-of-war between 3 different kinds of forces:


*

*One of these forces is the electromagnetic force. This force is due to the positive charges on the protons. Positive charges - or any electric charges of the same polarity - in close proximity, try to repel each other. This force, thus, wants to blow the nucleus apart.

*The other force is the residual strong force. This force is more difficult to explain, but basically it results in a very strong attraction of protons and neutrons to each other (in any combination), once they are suitably close together. This force wants to cram the nucleus together. However, the force decreases very quickly with distance - much more quickly than the inverse-square of the electromagnetic force, even though at short distances it far surpasses it in strength.

*The third force is very strange because, at least in this context, it's not what you usually think of as a "force" at all, i.e. something which pushes and pulls stuff around, but rather what it does is to change protons and neutrons into each other, and its aim is thus to "pull" their numbers toward a balance. This "force" is called the weak interaction.


Depending on how the balance of all three of these forces go, the nucleus with either be at an equilibrium, or stable, or it will not be, and thus unstable and the forces will act together to move it toward that equilibrium, changing it as may be such as by ejecting parts and converting protons/neutrons into each other until the equilibrium condition is achieved.
