Basically, I think the idea that the universe is infinite comes from considerations of the large-scale curvature of spacetime. In particular, the FLRW cosmological model predicts a certain critical density of matter and energy which would make spacetime "flat" (in the sense that it would have the Minkowski metric on large scales). If the actual density is greater than that density, then spacetime is "positively curved," which implies that it is also bounded - that is, that there is a certain maximum distance between any two spacetime points. (I don't know the details of how you get from positive curvature to being bounded, but as suggested by a commenter, look into Myers's theorem if you're curious.) However, if the actual density is not greater than that critical density, there is no bound, which means that for any distance $d$, you could find two points in the universe that are at least that far away. I think that's what it means to be infinite.
Overall, the observations done to date, paired with current theoretical models, are inconclusive as to whether the actual density of matter and energy in the universe is greater than or less than (or exactly equal to) the critical density.
Now, if the universe is in fact infinite in this sense, it still could have had a big bang. The FLRW metric includes a scale factor $a(\tau)$ which characterizes the relative scale of the universe at different times. Specifically, the distance between two objects (due only to the change in scale, i.e. ignore all interactions between the objects) at different times $t_1$ and $t_2$ satisfies
$$\frac{d(t_1)}{a(t_1)} = \frac{d(t_2)}{a(t_2)}$$
Right now, it seems that the universe is expanding, so $a(\tau)$ is getting larger. But if you imagine running that expansion in reverse, eventually you'd get back to a "time" where $a(\tau) = 0$, and at that time all objects would be in the same position, no matter whether space was infinite or not. That's what we call the Big Bang.