How can black holes be observed to grow?
Like a hailstone. See Kevin Brown's formation and growth of black holes. He says things like this:
"Incidentally, we should perhaps qualify our dismissal of the "frozen star" interpretation, because it does (arguably) give a servicable account of phenomena outside the event horizon, at least for an eternal static configuration. Historically the two most common conceptual models for general relativity have been the "geometric interpretation" (as originially conceived by Einstein) and the "field interpretation" (patterned after the quantum field theories of the other fundamental interactions). These two views are operationally equivalent outside event horizons, but they tend to lead to different conceptions of the limit of gravitational collapse. According to the field interpretation, a clock runs increasingly slowly as it approaches the event horizon (due to the strength of the field), and the natural "limit" of this process is that the clock asymptotically approaches "full stop" (i.e., running at a rate of zero). It continues to exist for the rest of time, but it's "frozen" due to the strength of the gravitational field..."
He doesn't favour this frozen-star interpretation, but if you read on, you realise that the alternative is where the infalling clock goes beyond future infinity into another region of spacetime. I don't buy that, so I prefer the frozen-star interpretation.
If, from the reference frame of an observer at rest outside a black hole, it takes an infinite amount of time for an object to be observed to fall into a black hole, how can black holes ever be observed to grow in that reference frame?
Very easily. The black hole is something like a hailstone. Water molecules can't pass through its surface, but they can alight upon its surface and be surrounded and buried by other water molecules. So the surface passes through them. So they effectively pass through the surface, and the hailstone grows. It's similar for infalling matter and the event horizon. They don't pass through the event horizon. The event horizon passes through them. When a star collapses, the event horizon grows from the centre out. You can see a mention of that in Friedwardt Winterberg's Gamma Ray Bursters paper.