Thanks to time dilation, a distant observer watching a man fall in to a black hole will only see him asymptotically approach the event horizon. So how do black holes ever get bigger?
In the frame of reference of the black hole (if there is such a thing), the man is indeed absorbed by the black hole. But the information will never reach the observer since any photon emitted past the event horizon (which is not the black hole surface) will never reach the observer.
I don't think there's a paradox here: The infalling observer will observe himself passing the horizon. The observer at infinity does not see this happen, but she will notice, with time, that the region from which she never receives signals has expanded.
This question crops up a lot, and I think it illustrates an interesting aspect of GR. Specifically what exactly does the question "How do black holes accrete mass?" mean?
In GR most questions don't make sense unless you specify what observer you're asking about or whether you want an observer independant reply. In the case of objects falling into a black hole the observer independant reply is easy because the geodesic of an object falling into a black hole is easy to calculate. See any introductory GR textbook, or if you want to ask a new question about this I'll be happy to post the details.
But most people asking the question have an observer in mind, and typically it's the Schwarzschild observer i.e. the observer sitting stationary at infinity (or far enough away to be effectively infinitely far away). In this case you're quite correct that this observer will never see an object reach the event horizon, but then the Schwarzschild observer will never even see the black hole form.
What the Schwarzschild observer can do is measure the curvature at some non-zero distance from the event horizon, and from this infer that the black hole must exist. If the observer does this, then throws some mass into the black hole and remeasures the curvature they will find it has increased. This allows them to infer that the mass of the black hole has increased i.e. the black hole has accreted mass.
It's tempting to get excessively philosophical and ask what we mean by "exist", but I think this temptation should be resisted as long as the observer independant calculation gives a clear result. If I do measurements of some object and those measurements are consistent with the Schwarzschild metric of a black hole then I think I'm justified in concluding that the object is a black hole.
Firstly, just considering the behaviour of infalling matter in a Schwarzschild metric doesn't directly answer the "how can they possibly grow?" question, since this metric describes an unchanging "eternal" black hole with constant mass. The other problem is that the traditional event horizon, which is often described as the defining feature of the black hole, only makes sense in the context of the entire picture of the spacetime, i.e. including its full time evolution. This is because the EH is defined as the boundary of the region in which photons are unable to escape to future null infinity, and we don't know for a fact that a photon can make such an escape until we have a picture of the entire spacetime laid out before us.
For this reason, people have considered other, more local (i.e. not requiring the whole of space-time), black hole definitions based on trapping horizons, and dynamical horizons. Here is a reference describing a dynamical horizon. In this ref fig. 4 shows a Penrose diagram of the Vaidya spacetime, in which infalling radiation causes the dynamical horizon to grow. Surprisingly, the traditional event horizon, however, reaches back even into the flat (blue) region. In the right hand picture, infall of radiation stops at $\nu=\nu_0$ and from then on the black hole looks like a Schwarzschild black hole.