Why do things decay in terms of base $e$? For instance, why do radioactive elements follow some $e^{-x}$ relationship?  Why not $\pi^{-y}$ or some other numerical constant as the base?  Is there some fundamental reason for Euler's number being the base of an exponential relationship for physical phenomena like radioactive decay, scale heights in atmospheres, etc.?
 A: $$\pi^x = e^{ln(\pi)x} = e^y $$
You can always write it as a power of e
A: In some sense, all bases are equivalent. If something decays as $\sim e^{-t/\tau}$, where $\tau$ is a decay constant, then it also decays as $\sim b^{-t/\tau_b}$ for any base $b$, where $\tau_b = \tau / \log_b e=\tau \ln b$ (which you can verify using properties of logs).
The reason we like to use base $e$ in physics is because it arises naturally in calculus. For example, $y(t) = y_0 e^{-t/\tau}$ is the solution to the differential equation
\begin{equation}
\frac{dy(t)}{dt} = -\frac{y(t)}{\tau}
\end{equation}
with the initial condition $y(0)=y_0$. In other bases, you would have an extra conversion factor appearing in the equation.
In fact in other fields where doing calculus isn't as important, it is more common to use other bases that are considered more "intuitive", like $10$ or $2$.
A: There's nothing special about $e$ except that it's convenient in the sense that $e^t$ is its own derivative.  The description of exponential decay arises from the model that the rate of change is proportional to the amount of stuff remaining, i.e.,
$$
\frac{dN}{dt} = -\gamma N\,,
$$
which is solved the most succinctly and easily by
$$
N(t) = N_0e^{-\gamma t}\,.
$$
However, we can change the base of the exponential to be anything we like, and we often do to get things like half-lives. That is,
$$
N(t) = e^{-\gamma t} = 2^{\log_2(e^{-\gamma t})}
=2^{-(\gamma\log_2(e))t}
\,.
$$
