Yes, for the harmonic oscillator, the amplitude diverges with time at resonance, as already discussed in Undamped Resonance of a Classical Harmonic Oscillator.
In general, what happens away from resonance depends both on the system (e.g., harmonic oscillator or anharmonic?), on the forcing (very high or low frequency, non-periodic?), and possibly on the starting point of the system. For example, the movement can be chaotic for the cubic (anharmonic) potential
$$ U(x) = \alpha x + \beta x^3, $$
which is the well-known Duffing oscillator.
Generally, the amplitude away from resonance will not diverge for small but nonzero damping, but will also not be necessarily constant; it'll grow during some oscillations, and decrease with others. When the forcing injects energy (i.e, in the instants its work is positive - when the force is in the same direction as the oscillation speed), then the amplitude increases, while decreasing otherwise (negative work done by forcing, which happens when it acts against the momentary movement).
The sinusoidally forced harmonic oscillator will, after a transient behavior, oscillate at the forcing frequency at constant amplitude. If the forcing is periodic, but not sinusoidal, or the oscillator not harmonic, then the steady-state amplitude is constant, but the movement is described by an infinite sum of sines (at multiples of the forcing frequency, i.e., a Fourier series). For example, Keith Fratus notes gives, for an arbitrarily forced harmonic oscillator, the following solution:
$$ x(t) = \sum_{n=1}^\infty \frac{\sqrt{a_n^2+b_n^2}}{\sqrt{(\omega^2-n^2\lambda^2)^2+4\beta^2n^2\lambda^2}}\cos{(n\lambda t - \delta_n-\phi_n)}, $$
where $\lambda$ is the angular frequency of the forcing, $\omega$ the oscillator frequency, and $\beta$ the damping constant.
Related discussions are A conceptual doubt regarding Forced Oscillations and Resonance and Physical reason behind having greater amplitude when driving frequency$ < $ natural frequency than that when driving frequency $>$ natural frequency.