As things approach the horizon of a black hole, their time dilation approaches infinity. How then can we observe black holes gaining mass if, according to our reference frame, nothing can pass the horizon of a black hole?

Even more so, in order for a singularity to form, material must compress until there is a horizon, which should take an infinite amount of our time to happen, again due to time dilation. Since we can observe black holes and they can gain mass, this is obviously flawed. How so?