We "know" that the neutrons are in the nucleus because they add to the weight of the atom. This is particularly visible in the case of isotopes. If you have some 12C (carbon with 6 protons and 6 neutrons in the nucleus) and some 13C (carbon with 6 protons and 7 neutrons in the nucleus), then both are definitely "carbon" (they engage in the same kind of reaction) so they have the same number of electrons, hence the same number of positive charges in the nucleus (since an atom is electrically neutral); but the 13C is definitely heavier (by about 8.3%). Thus, there must be some extra component in the nucleus, that does not add to the charge but has a definite mass.
Historically, the discovery of the neutron came first from an experiment from Rutherford, who was throwing alpha particles (helium nuclei) at a gold foil, and measuring how the particles were scattered. The scattering required that the gold nucleus was substantially heavier than the mass of its protons.
Later on we got a lot more data on neutrons, allowing us to not only be quite certain of their presence, but also measure a lot of their properties.
Within the nucleus, protons and neutrons stick together. The protons, being all positively charged, would very much like to fly apart from each other; but they are bound by the strong interaction which is much stronger than the electromagnetic repulsion (but with a very short range). That force also works on neutrons, which is why they stick (though, having a neutral charge, they are not so eager to leave).
A neutron, by itself, is not stable. It is slightly heavier than a proton, and, left alone, decays into a proton+electron after a few minutes (and an antineutrino). Within a stable nucleus, neutrons cannot decay because there is no room for an extra proton with the quantum state that would result from such a decay. When the nucleus has too many neutrons with regards to its number of protons, it is not stable, meaning that a neutron can morph into a proton, emitting an electron in the process (this is called beta decay).
Thus, neutrons stick to the nucleus because of the strong interaction, and they stay there indefinitely because the nucleus structure stabilizes them, preventing them from decaying.