As I understand it, we have a Brillouin zone in the primitive cell, in this primitive cell we only have one lattice point.
This already has a huge misunderstanding that makes it impossible to understand anything else, so we have to attack it first.
When you are talking about lattice points and primitive cells, what you are talking about are cells in real, position space. You have ions and electrons in this space.
Brillouin zone is in reciprocal space. Another name is momentum space. It is the space of reciprocal of wavelength, i.e. $\frac1\lambda$. Position space, you measure in units of length. Reciprocal space is measured in reciprocal of units of length. Totally different things. You do not find lattice points in reciprocal space.
What you do find there, are the parametrisation of the waves allowed by the symmetries of the lattice and the arrangement of ions and electrons in the unit cell. And even better, it is arranged in order of increasing energy of excitation, so you can get great understanding of the system by studying this way. It should be obvious, since this is a parametrisation of waves, then, whenever you want to study waves with regards to a system, then you really do want to work in reciprocal space. X ray diffraction for crystallography, electron waves themselves, phonons, and so forth, basically require an understanding of reciprocal space to really make sense thereof.
Unlike position space, where there is no privileged point to pick as the origin, the origin in reciprocal space is rather special, and that is, by convention, called the $\Gamma$ point. Because you have a lattice in position space, after conversion by Fourier transform, you get another lattice in reciprocal space. The smallest cell in reciprocal space, made by, say, the Voronoi method, is called the Brillouin zone, BZ, and if it is the one centred on $\Gamma$, that is called the $1^\text{st}$ Brillouin zone, usually abbreviated as 1BZ. If you just have the 1BZ and also a basis of reciprocal lattice vectors that connect $\Gamma$ to its nearest neighbours, then you can study the entire reciprocal space in a very efficient, simple, and nowhere-tedious manner. You might work out the electron waves in this manner, and thereby get exact solutions for the entire system.
Because the 1BZ and the set of all reciprocal lattice vectors $\{\vec G_i\}$ only covers the symmetries of the lattice points, if the unit cell you are starting with has some additional symmetries, then there will be symmetries in the 1BZ too. For example, it is common for waves travelling rightwards, say, to just be mirror images of waves travelling leftwards. So, if you have obtained the electron wavefunction that corresponds to a certain rightwards moving wave, then with some possible need to mutate the wavefunction, you would be able to deduce the electron wavefunction that corresponds to a leftwards moving one. This way, you can cut down on computational expenses, by just studying a subset of the 1BZ. The minimal subset of the 1BZ that you absolutely need to study fully, in order to characterise the entire system completely, is called the IBZ, that you know.
There is a lot more nice things to talk about the reciprocal space, but this is a good place to stop.