Is half-life a statistical average of variable decay times? Is the half life of a material only accurate as long as you are still in a macroscopic regime? If I had 8 particles in a box would I observe a fluctuation in half lives, and what would occur within the 4th half life?
 A: Half life is, by definition, the amount of time until half of an infinitely large sample would decay. That's precisely equivalent (according to the frequentist interpretation of probability, if that matters to you) to the time until an individual particle's probability of decay reaches one half. The half life is a theoretical quantity that doesn't depend on the actual number of particles you're dealing with.
If you actually put 8 particles in a box and watch how long it takes for half of them to decay, you could consider that a measurement of the half life of the particles. As with any measurement, the value you measure will not, in general, be the same as the true (theoretical) value. So yes, there will be fluctuations, and once the number of particles remaining drops to two or one or zero, those fluctuations will be very very large. But what is fluctuating is your measurement of the half life, not the true theoretical half life itself.
A: Yes, it is a statistical average in the sense that the measured half life will approach a single value of a true half life if you do lots of measurements.
In other words, if you did the experiment many, many times you would find that on average you had 4 particles left after a half-life had passed.
For any individual experiment, the results would vary.
Each atom has a probability of surviving intact after a time $t$ according to 
$$p = \exp(-\lambda t)$$
where $\lambda$ is the decay constant and the half life $t_{1/2} = \ln 2/\lambda$.
If you wait 4 half lives then $t = 4\ln 2/\lambda$ and the probability of an individual particle surviving is $\exp(-4\ln 2) = 0.0625$. 
In practice, you have to have an integer number of particles, so the most likely outcomes are either 1 or zero intact atoms remain.
If you have 8 atoms and the probability that any of them will have decayed is $p=0.0625$, then one can use the binomial probability distributionto work out the probability that any number $n$ will survive from a population of $N$ is
$$ P(n) = \frac{N!}{n! (N-n)!} p^{n}(1-p)^{N-n}$$
So $P(0)= 0.597$, $P(1) = 0.318$, $P(2)= 0.037$ and so on.
Now, if your aim is to estimate the half life based on a single experiment with these 8 atoms, then I see (at least) two possibilities.
(i) If you measure the time it takes for the 4th decay to occur, then you can calculate $P(4)$ as above, but calculate it for a range of possible values of $\lambda$. This will give you a probability distribution for $\lambda$ from which you can find the maximum likelihood value or a confidence interval.
(ii) If you have the individual decay times of each decay, then for each atom you can calculate a probability that it would have decayed in less than its observed decay time, given an assumed $\lambda$, which is $P_i(\lambda) = (1- \exp[-\lambda t_i])$. You can also include any atoms that haven't decayed, $P_i(\lambda) = \exp[-\lambda t_i]$. You then form the product of these probabilities $P(\lambda)= \prod P_i(\lambda)$ to give you an overall likelihood distribution for $\lambda$, from which you can estimated a maximum likelihood value for $\lambda$ and a confidence interval.
