Does average lifetime even mean anything? So today I was trying to derive an expression for the number of radioactive atoms remaining after a time $t$ if I began with $N_0$ atoms in total. 
At first I tried to assume that they had an average lifetime and work from there, but my friend dropped a hint and I found it much easier to assume that each atom had a specific chance in a small time element $dt$ to decay. 
After some manipulation, I arrived at an exponential decay formula, which was great. However, it got me to thinking about the concept of 'average lifetime'. (I also managed to find an expression for average lifetime relating it to the chance a single atom will decay in a differential time element). 
If I had a group of atoms that have an 'average lifetime' of say 5 seconds, after 5 seconds has elapsed, what is the 'average lifetime' of the remaining atoms? 
I don't think I can arbitrarily choose some reference time to begin ticking away at the atoms' remaining time, does that mean at any point of time that their 'average lifetime' or expected lifetime is always a constant, and never actually diminishes as time goes on?
 A: What is the "average life time"? You take the average of the life times of many identical atoms, i.e.
$$
T_{\text{avg}} = \frac{1}{N}\sum_{n=1}^{N} T_n
$$
This is a random variable dependent on your distribution of $T_n$, which is in your case the exponential function: $p(T_n = t) = \alpha e^{-\alpha t}$ if $t\ge 0$ and zero otherwise. Knowing that, you can calculate the distribution of the "average life time" as
$$
p(T_{\text{avg}}) = \int_0^\infty dt_1 \dots \int_0^\infty dt_N\ \delta\!\left(t - \frac{1}{N} \sum_{n=1}^{N} t_n \right)\ \prod_{n=1}^{N}p(T_n = t_n)
$$
$$
= N\int_0^\infty dt_2 \dots \int_0^\infty dt_N\ p\!\left(T_1 = Nt-\sum_{n=2}^{N}t_n\right)\ \prod_{n=2}^{N}p(T_n = t_n)
$$
Here, the argument of $p(T_1 =\dots)$ can become less than zero, so you must clip the upper integral boundaries.
$$
\dots = N\int_0^{Nt}dt_2 \int_0^{Nt-t_2}dt_3 \dots \int_0^{Nt-\sum_{n=2}^{N-1}t_n}dt_N\ \alpha e^{-\alpha (Nt - \sum_{n=2}^{N} t_n)}\ \prod_{n=2}^{N} \alpha e^{-\alpha t_n}
$$
$$
=N\alpha^N e^{-\alpha Nt} \int_0^{Nt}dt_2 \int_0^{Nt-t_2}dt_3 \dots \int_0^{Nt-\sum_{n=2}^{N-1}t_n}dt_N\ 1
$$
$$
=N\alpha^N e^{-\alpha Nt} \int_0^{Nt}dt_2 \int_0^{Nt-t_2}dt_3 \dots \int_0^{Nt-\sum_{n=2}^{N-2}t_n}dt_{N-1}\ \left(Nt-\sum_{n=2}^{N-1}t_n\right)
$$
$$
=N\alpha^N e^{-\alpha Nt} \int_0^{Nt}dt_2 \int_0^{Nt-t_2}dt_3 \dots \int_0^{Nt-\sum_{n=2}^{N-3}t_n}dt_{N-2}\ \frac{1}{2}\left(Nt-\sum_{n=2}^{N-2}t_n\right)^2
$$
$$
\dots
$$
$$
=\frac{N}{(N-1)!}\alpha^N (Nt)^{N-1} e^{-\alpha Nt}
$$
For very large $N$, this becomes a very sharply peaked function, and the peak is where the derivative w.r.t. $t$ becomes zero:
$$
T_{\text{avg}} = \frac{1}{\alpha} \frac{N}{N-1} \approx \frac{1}{\alpha}
$$
This is the average life time of your isotopes. Note, that this is equal to the expected life time of one single isotope:
$$
t_{\text{expected}} = \int_0^\infty dt\ t\cdot p(T_1 = t) = \int_0^\infty dt\ t\alpha e^{-\alpha t} = \frac{1}{\alpha}
$$
which is probably what you meant by that. The expected life time of an atom does not change over time. Imagine you expect a life time of 1 minute when you start to observe a single atom. Then, when half a minute has passed, what further life time do you expect? Still a full minute. This might seem strange, but compare it to other games of chance.
For instance, if you toss a coin and it comes up heads, you should not expect the coin to come up tails the next toss. You should not even think it slightly more probable. The second throw has exactly the same chances as the first one.
The same with the atoms. It does not matter when you start to observe the atom, it has always the same expected life time, $\frac{1}{\alpha}$, and it does not even change as you are observing it.
A: Congratulations on deriving the exponential law for yourself, one learns a great deal about science working like this. Now to your last question:

If I had a group of atoms that have an 'average lifetime' of say 5 seconds, after 5 seconds has elapsed, what is the 'average lifetime' of the remaining atoms? I don't think I can arbitrarily choose some reference time to begin ticking away at the atoms' remaining time, does that mean at any point of time that their 'average lifetime' or expected lifetime is always a constant, and never actually diminishes as time goes on?

Yes indeed the average lifetime is constant. And the exponential distribution you have derived is the unique lifetime distribution with this property. Another way of saying this is that the decaying particle is memoryless: it does not encode its "age": there is nothing inside the particle that says "I've live a long time, now its time to die". Yet another take on this - as a discrete rather than continuous probability distribution - is the geometric distribution of the number of throws before a coin turns up heads, and the observation that a coin has no memory that counters the famous gambler's fallacy. 
To understand this uniqueness, we encode the memorylessness condition into the basic probability law 
$$p(A\cap B) = p(A) \, p(B|A)$$
Suppose after time $\delta$ you observe that your particle has not decayed (event $A$). If $f(t)$ is the propability distribution of lifetimes, then the probability the particle has lasted at least this long, i.e. the probability that it does not decay in time interval $[0,\,\delta]$ is: 
$$p(A) = 1-\int_0^\delta f(u)du$$
The a priori probability distribution function that the particle will last until time $t+\delta$ and then decay in the time interval $dt$ (event $B$) is 
$$p(B\cap A) = f(t+\delta) dt$$. 
This is events $B$ and $A$ observed together, which is the same as plain old $B$ since the particle cannot last unti time $t + \delta$ without living to $\delta$ first! Therefore,  the conditional probability density function is
$$p(B|A) = \frac{f(t+\delta)\,dt}{1-\int_0^\delta f(u)du}$$ 
But this must be the same as the unconditional probability density that the particle lasts a further time $t$ measured from any time, by assumption of memorylessness. Thus we must have:
$$\left(1 - \int_0^\delta f(u)du\right)\,f(t) = f(t+\delta),\;\forall \delta>0$$
Letting $\delta\rightarrow 0$, we get the differential equation $f^\prime(t) = - f(0) f(t)$, whose unique solution is $f(t) = \frac{1}{\tau}\exp\left(-\frac{t}{\tau}\right)$. You can readily check that this function fulfills the general functional equation $\left(1 - \int_0^\delta f(u)du\right)\,f(t) = f(t+\delta)$ for any $\delta > 0$ as well.

As Akhmeteli's answer says, true memorylessness is actually incompatible with simple quantum models. For example, one can derive the exponential lifetime for an excited fluorophore from a simple model of a lone excited two state fluorophore equally coupled to all the modes of the electromagnetic field. The catch is that the derivation rests on approximating an integral over positive energy field modes by an integral over all energies, both positive and negative. This of course is unphysical, but an excellent approximation since only modes near to the two state atom's energy gap will be excited: the fluorophore "tries" to excite all modes equally, but destructive interference prevents significant coupling to modes of greatly different energy than the difference between the energies of the states on either side of the transition. 
I show how this analysis is done in this answer here  and here.
Linewidths are mostly extremely narrow compared to the frequencies of the photons concerned, so I find it surprising and quite wonderful that Ahkmeteli cites a paper giving experimental evidence of the nonconstant lifetime.
A: You are right, the average life time remains the same. In the context of your example, if you have $N$ nuclei at any arbitrary point of time $t_0$ and if $T$ is the half life of the nucleus then at time $t_0 + T$, half of them would have decayed. 
That $T$ is independent of when you started keeping time is the key observation in making carbon dating a reliable tool to estimate age of fossils.
A: Khalfin showed some 60 years ago that strictly exponential decay is actually incompatible with quantum theory and there must be tiny deviations both for very small and very long times. See the details and references, say, in Nature vol. 335, p. 298 (22 September 1988). There seems to be experimental confirmation as well: http://dro.dur.ac.uk/4234/1/4234.pdf (PRL 96, 163601, 2006). So, strictly speaking, life-time cannot be constant.
EDIT (04/28/2015): the proof can be found, e.g., at http://www.ias.ac.in/pramana/fm2001/QT4.pdf (Pramana - journal of physics, vol 56, pp. 169-178). The proof uses the fact that the spectrum of the Hamiltonian is bounded from below and the .Paley-Wiener theorem
