The one thing to keep in mind is that in order to perform a gravity-assist maneuver, you need to be able to enter a hyperbolic orbit around a given body that is moving relative to your destination. And, in order to be in such an orbit, there is a specific range of velocities for every object that you must have (dependent on mass of the object). So the fastest you can get to by gravity-assist is much less than relativistic speeds because at relativistic speeds, you would not be able to enter into a proper hyperbolic orbit.
It is true that at any high speed, a flyby constitutes a hyperbolic orbit; however, to use a gravitational slingshot, you need to enter against the object's motion and exit with the motion from that object's point of view. At relativistic speeds and for most regular bodies, your orbit would closely resemble a straight line, there could be no gain of velocity.
A good gravity assist works if you can ensure that your hyperbolic trajectory minimizes the angle $\theta$ between the assisting body's trajectory and the spacecraft's exit trajectory. It is given by:
Where $e$ is the eccentricity of the orbit and must satisfy $e\geq1$. From this, one can see that a parabolic trajectory is best as the exit trajectory is directly in line with the body's trajectory. We can also see that as $e\rightarrow\infty$, the exit trajectory is at right angles to the body's trajectory and we get no help from the assist. Additionally, as your velocity increases, it will force $e$ to become larger unless you significantly increase the mass of each subsequent object. So, for a normal assisting body, like a star or a planet, travelling past it at relativistic speeds will result in minimal orbital deviation; there will be practically no transfer of momentum, which makes for an even smaller increase in velocity.
The fastest a spacecraft can get to using gravity-assists very much depends on the largest mass of the objects you use. However, I cannot give you an estimate of a number because due to the sheer impracticality of using gravity-assists to achieve extreme velocities, we (rocket scientists) haven't ever tried computing a theoretical limit. I can guarantee you though that without using high density objects (neutron stars, black holes, etc.), no spacecraft will reach velocities near the speed of light by gravity slingshots alone.