Yes, what you're thinking of is known as the stochastic gravitational wave background. If it is present, it would be exactly like the CMB in the sense that it would be (almost) uniform across the whole sky and would essentially just be noise. However, disentangling the contribution from inflation (the period of accelerated expansion in the very early Universe) from the contributions from compact object mergers later in the Universe's history is not simple. The detectability of the background depends on its frequency, which in turn depends on the source of the gravitational waves.
In the figure below, I used the excellent tool at gwplotter.com to plot some different gravitational wave sources and current and future detectors. The stochastic background is the orange band at low frequency, accessible only to pulsar timing arrays (need many years to collect data) and the upcoming Square Kilometre Array (SKA).
The upcoming space-based detector LISA is shown, probing the mid-frequency band in blue, which constitutes massive binary objects and extreme mass ratio inspirals (solar mass black holes orbiting super-massive black holes, for example).
Advanced LIGO (aLIGO) is shown, along with the now commonly detected compact object mergers in red. Also at that end of the frequency range is the Einstein Telescope, a planned ground-based detector which will have three arms rather than two.
A better way of directly probing inflation comes from the potential observation of B-mode polarisation in the CMB (which was famously falsely detected in 2014 by BICEP). Primordial B-modes can only be sourced by tensor perturbations (i.e. gravitational waves from inflation), so if we were to observe them in the CMB it would be an inflationary smoking gun. However, secondary B-mode polarisation can be generated by weak lensing and absorption of CMB photons by dust as they propagate towards us, and as yet, no definitive primordial B-modes have been observed.