$\let\D=\Delta \let\lam=\lambda$
What @anna v says, her quotation included, is all OK, but for a
fundamental point. It is where she writes
It is the simplest logical assumption, assuming constant depletion by the $\lam$ per unit time, to start describing statistical depletion.
To illustrate my point I choose to take a special example: the decay of $^{226}\mathrm{Ra}$, an isotope of radium, $\alpha$-decaying with half-life 1600 years. The first question someone novel to the field should ask is: how so long a time span could be measured? The answer resides in one formula @anna v quotes:
$$\D N = -\lam\,N\,\D t.\tag1$$
Assume you are given a sample of substance where you are said there is some radium. If you are an expert chemist you will be able to make a quantitative analysis, in particular to determine the total mass of radium contained in the sample. Then, knowing radium molar mass and Avogadro's constant, you will calculate $N$ (I don't give details, as they are inessential to our purpose).
Now you - as a clever experimental physicist - setup the equipment to count the number $\D N$ of Ra-nuclei that will decay in a certain time $\D t$. Once you hold $N$, $\D t$ and $\D N$ eq. (1) gives $\lam$, and you are done. Well not quite: relation between $\lam$ and half-life is simple, but requires you are familiar with exponentials and logarithms. Are you? It doesn't matter. Content ourselves with a rough idea: half-life is not far from $1/\lam$.
My aim was simply to show how it is possible to determine a half-life of 1600 years without expecting a comparable time. It is possible because $N$ will likely to be an extremely high number. Just to give an example: in a microgram of $^{226}\mathrm{Ra}$ there are about $2.7\cdot10^{15}$ nuclei (almost 3 quadrillions). So also in a small fraction of half-life you will observe a substantial number of decays, and this makes the measurement a feasible affair.
But the above was only a preparation for the real problem. Our friend, a newcomer to radioactivity, has been satisfied as to his first question, and has another ready. He asks "Where does radium come from? I ask this, because 1600 years are a long time on human scale, but not for many natural processes. If we can find some radium around, it must have been created - at most - a few thousands years ago. I heard radium is found in very ancient minerals, maybe billion years old. How did radium enter those rocks in relatively recent times?"
The teacher knows the answer and replies: "Very good question.
Actually radium belongs to a 'family' of radioactive elements, whose 'father' is $^{238}\mathrm U$ (uranium). After a chain of several decays I will not detail, radium is born. The point is that the father has a very long half-life: 4.5 billion years. We know that $^{238}\mathrm U$ was generated well before Earth formation, in a primordial supernova explosion. Debris of that explosion formed the cloud whose gravitational collapse gave origin to solar system, about 5 billion years ago. So, even if the subsequent decays are relatively fast, there is always enough uranium to replenish the chain."
An immediate reply follows: "I see, but your explanation raises a serious doubt. If radium nuclei are being continuously created anew, I expect that in my sample there are nuclei of all ages. How can they possibly all follow in their decay the same law? Shouldn't they in some sense 'remember' their birth date?"
Now I'll stop my "fiction" and speak myself. The problem is the
following. It is reasonable to assume that all radium nuclei are born in the same state, at different times. They will evolve from that state according to the same law, but each from a different time origin. How can we explain that - as far as we can see - their time evolution (which we see as a decay probability) remains the same starting from present time, if present time means a different previous time span of their life?
I don't want to write equations, but this is a real problem for
quantum mechanics. It is not trivial that it has actually been solved in the affirmative, bat with some caveats. It is generally true that the probability of finding an undecayed nucleus decreases exponentially with time. But this is not true
for $t$ very small or very large.
Unfortunately mine is a distant memory and I'm not able to give references. Maybe some reader can help?