Free neutrons pass through most materials but how do they damage materials? Neutrons has no charge so they are hard to interact with materials. That's what I'm thinking.
But they collide with atoms and it causes nuclear fission.
And they can also be a dangerous radioactive which bring out biological damage.
Why? You know that the most of the atom is empty space.
 A: Well you have two effects.
For starters, free neutrons are unstable, and they decay to a proton and an electron, with a half life of about 11 minutes.   So then you have two charged particles that fly off in opposite directions wreaking havoc.
Secondly, neutrons, being uncharged, are not deflected by Coulomb forces, so they can score a direct hit on the atomic nucleus, and be captured.  This then usually results in a proton being ejected from the nucleus, so the atom changes to a different element, of one less atomic number.
The ejected proton ("knock on" proton) also has high kinetic energy; often around 14 MeV, so it is the energetic charged particle products, that do the damage.
Neutron damage in biological tissue, is a strong function of neutron KE.  Special "tissue equivalent" monitor detectors, are used to monitor neutron hazards, in locations where they could be produced. Impacting Deuterons, on a heavy ice target, will result in neutrons, or protons, in the 14 MeV range.   The detectors, often proportional gas counters, containing organic materials either solid or gas, or both, that collectively mimic the capture cross-section for neutrons in biological tissues.  These detectors, are like Geiger counters, but below the critical avalanche Voltage range
A: Atoms are very small, you are correct. But if you have enough neutrons, the probability of at least one hitting what it needs to hit increases dramatically.  That is why a nuke goes off when it reaches a critical mass (where there are enough neutrons produced to reliably trigger additional collitions). It is also why you body has a hard time defending against them as they can pass through most tissue and randomly hit DNA or other important components.  Again, most pass through as you are correctly predicting but if you are exposed to enough of them, the damage that the minority of them cause builds up.
A: they interact purely by linear energy transfer (LET. think billiard ball collisions) since they are chargeless. because of this neutron "radiation" is only indirectly radioactive. 
when free neutrons strike fissile material unstable isotopes form, which quickly decay, producing ionizing radiation and sometimes yet more free neutrons--a neutron chain reaction if you will. sometimes the struck material doesnt have to be radioactive itself, but turns radioactive upon neutron bombardment, such as cobalt-59(present in stainless steel) turning into dangerous cobalt-60 from neutron capture. 
but if you are talking biological damage it would make sense to talk more about hydrogen and oxygen, of which we are full of (in the form of water). in the former case, LET effectively kicks the proton and turns the hydrogen nucleus (we are full of hydrogen in the form of water) into a free proton which can directly ionize and severely damage the body from within. oxygen-16 releases a gamma photon upon neutron capture, and quickly ejects a free proton to form nitrogen-16. within seconds this decays back to O-16 by beta decay. nothing that was released as byproducts in these processes are good for the body. most importantly, the danger of indirect radioactivity in biology is ionizing irradation from inside the body rather than from outside.
