In principle, a single triple black hole system could be used to accelerate a probe to relativistic velocities.
Consider the following configuration: two black holes (of comparable masses), let us denote them $A$ and $B$, are closely orbiting each other on nearly circular orbits, while the third black hole, $C$ orbits around the first two at much greater distance and thus with lower velocity (see the image):
A probe starts from vicinity of $C$ towards $A$ and $B$ and after circling one of this pair flies back toward $C$, circles it performing almost perfect turnover and flies back again towards $A$ and $B$ and so forth. During each time the trajectory passes near the component of pair $A$ and $B$ that has instantaneous velocity component directed toward $C$. As a result of each flyby near $A$ or $B$ the probe gathers speed. If all velocities remain nonrelativistic, the speed gain is approximately double the instantaneous velocity component of either $A$ or $B$ toward $C$. Taking into account that usually instantaneous velocity $\mathbf{v}_A$ or $\mathbf{v}_B $ would be at some angle from direction to $C$, on average after large number $N$ of iterations the probe's velocity could be of order $v_\text{probe}\sim N u$ where $u$ is an average orbit velocity of a pair $A$ and $B$.
As probe's velocity become comparable with the speed of light, relativistic laws of velocity addition must be taken into account, so of course the probe could never go faster than the speed of light, but (in principle) could approach it after sufficiently large number of iterations. For a triple black hole system described here, similar trajectories could be constructed for photons, they would of course be always travelling at the speed of light but as a result of this gravitational assists photons would gain energy, extracting gravitational/kinetic energy from the system in, these trajectories could be seen as ultimate limit of gravitational assits.
If instead of black holes we use other bodies such as white dwarfs, neutron stars or even ordinary stars, then the type of gravitational assist described here would have a builtin speed limit in the form of escape velocity on the surface of a body. The lowest of such velocities would determine the maximum velocity of a probe that completes almost complete turnover near such body (for bodies like neutron stars, that need GR to describe orbits around them, the relationship between surface escape velocity and orbital velocity is nontrivial). This limit explains why this type of maneuver would not work for small planets: escape velocity of e.g. Mercury is $4.25\,\text{km/s}$, while Mercury's orbit velocity is $47.4\,\text{km/s}$, so Mercury cannot turn around a probe so that it can fully benefit from its large orbit velocity.
Of course, there are a lot of technical issues that could complicate and impose limitations for such type of travel: tidal accelerations on a probe near a black hole or neutron star could be very large, the maneuvering must be very precise, especially as velocities grow, etc.
Another type of “limit” for gravitational assists has been considered by Freeman Dyson in a paper
- Dyson, Freeman. Gravitational machines. in Interstellar Communication, edited by AGW Cameron, (Benjamin Press, New York, 1963) (1963), free pdf.
This is a limit not in terms of velocity but in terms of scale: if some civilization surrounds e.g. a pair of white dwarfs orbiting each other by streams of masses extracting gravitational and kinetic energy via gravitational assists, then potentially the power extracted could exceed solar luminosity by several orders of magnitude.