First, the speed of other galaxies isn't too helpful. For example, the radial velocity of the Andromeda galaxy relatively to us is 300 km/s, i.e. 0.1% of the speed of light only. Moreover, internally, everything in that galaxy moves by pretty much the same speed and is confined to the vicinity of that galaxy which makes us pretty sure that no piece will reach us before Andromeda galaxy will.
More importantly, macroscopic systems in outer space aren't moving with the same huge speeds near the speed of light as the cosmic rays essentially because of the second law of thermodynamics.
When cosmic rays are accelerated to high speeds, we may treat them statistically and the high energy of these elementary particles may be assigned a high temperature. But the physical systems with many degrees of freedom prefer to evolve to the most likely, high-entropy configurations. That's why the excess energy (e.g. in a supernova) tends to divide between the elementary particles chaotically.
In particular, the individual particles' kinetic energy results from velocities that have a random direction. At these huge temperatures (kinetic energies per particle), the atoms are unbound (well above the ionization energy) and macroscopic matter doesn't exist. So the likelihood that a large object will move towards the Earth at a near-luminal velocity is as unlikely as the possibility that the numerous atoms in the large objects are assigned velocities with the exactly equal direction although the directions are being chosen from an isotropic, random distribution.
After some time for "thermalization" (interactions between atoms are allowed to change the system with the conservation laws' being the only absolutely constraints), the greater object you consider, the less likely it is that all the atoms in that object will be doing the same thing. This is a form of the second law of thermodynamics.
The previous paragraph prevents the creation of "coherent macroscopic cosmic rays", macroscopic objects that would move in the same direction, from the thermal chaos of hot environments such as the supernova. But even if some astrophysical process managed to eject a chunk of matter at these high speeds, the previous paragraph will still guarantee that we won't receive it on Earth. Instead, the individual atom of that speedy object would still have some residual mutual velocities so the object would split into individual atoms and we would observe cosmic rays only once again.
I could summarize the situation in this way: to shoot a large object by a huge speed from a very distant celestial body to the Earth requires one to have the same and huge radial velocities of all atoms but almost vanishing relative transverse velocities. The likelihood of that goes to zero exponentially in any thermal environment.
If one assumes that the source of the initial speed is not thermal, then one must accept that the projectiles will derive their speed from their broader environments – speeds of astrophysical bodies that already exist – and those are simply of order 0.1 percent of the speed of light as in the Andromeda example or lower. These modest speeds boil down to the inhomogeneities during the structure formation and, ultimately, to inflation, or to the planetary speeds derivable for a star. Whenever you want to locally violate the modesty of the speeds, e.g. by a gravitational collapse, you don't "shoot" any new particles before the collision and when the collision happens, the excess energy of the collision also inevitably creates a high temperature and we're back to the previous paragraph.
So near-luminal macroscopic bodies aren't observed. I would even go further and despite my being a believer that life in the Universe is extremely rare, I would say that an object moving by a near-luminal speed would prove the existence of an advanced civilization. I should be a bit careful: the gravitational slingshot is a process that allows the speed to be enhanced even naturally. But even if the source of the speed were a gravitational slingshot and the speed would be really high, like 99.9999% of the speed of light, chances would be high that some intelligence was behind the optimization of the gravitational slingshot because it's extremely unlikely for such an outcome to occur naturally.
