The one thing to keep in mind is that in order to perform a gravity-assist maneuver, you need to be able to enter a hyperbolic orbit around a given body that is moving relative to your destination. And, in order to be in such an orbit, there is a specific range of velocities for every object that you must have (dependent on mass of the object). So the fastest you can get to by gravity-assist is much less than relativistic speeds because at relativistic speeds, you would not be able to enter into a proper hyperbolic orbit.
It is true that at any high speed, a flyby constitutes a hyperbolic orbit; however, to use a gravitational slingshot, you need to enter against the object's motion and exit with the motion. At relativistic speeds and for most regular bodies, your orbit would closely resemble a straight line, there could be no gain of velocity.
A good gravity assist works if you can ensure that your hyperbolic trajectory minimizes the angle $\theta$ between entry and exit. It is given by:
Where $e$ is the eccentricity of the orbit and must satisfy $e\geq1$. Additionally, as your velocity increases, it will force $e$ to become larger unless you significantly increase the mass of each subsequent object.
The fastest a spacecraft can get to using gravity-assists very much depends on the largest mass of the objects you use. However, I cannot give you an estimate of a number because due to the sheer impracticality of using gravity-assists to achieve extreme velocities, we (rocket scientists) haven't ever tried computing a theoretical limit. I can guarantee you though that without using high density objects (neutron stars, black holes, etc.), no spacecraft will reach velocities near the speed of light by gravity slingshots alone.