This is really just an elaboration of Ben's comment (Ben, if you want to convert your comment to an answer I'll delete this).
As long as you are more than a few Schwarzschild radii away from the black hole it's gravitational field is indistinguishable from a star of the same mass. We already know stars can be ejected from galaxies at large velocities, so that means exactly the same applies to black holes.
Stars have huge separations relative to their radii, so the chance of any two stars colliding is vanishingly small. That means the ejected star/black hole is very unlikely to get close enough to another star for the difference between the star and black hole to be detectable. In both cases there would be the same gravitational perturbation of any system the ejected body passed.
As far as I know we have never observed any hypervelocity black holes, but then this would be a hard thing to observe. The black holes we know about tend to be in binary systems where acretion of gas from the partner star causes X-ray emission. We can tell the radiating object is a black hole because we can infer the black hole mass from the dynamics of the binary. With an isolated black hole we would have no way of determining it's mass, so we couldn't tell it from e.g. a neutron star. In fact, without a nearby partner to acrete gas from we probably couldn't see the black hole at all because it wouldn't radiate enough energy.