On a (pseudo-)riemannian manifold $M$, for a conservative vector field $J^\mu , \ \nabla_\mu J^\mu = 0$ (a "conserved current"), we have for its flux through the boundary of any submanifold $S$:
$$
\int_{\partial S} J^\mu n_\mu \mathrm{d}\mathbf{\sigma} = \int_S \nabla_\mu J^\mu \mathrm{d\mathbf{vol}} = 0
$$
Where $n^\mu$ is the (outward-pointing) normal to the hypersurface $\partial S$ and $\mathrm{d}\mathbf{\sigma}$ is the volume form induced on it by the volume form $\mathrm{d\mathbf{vol}}$ of $M$ (which comes from the metric).
Then, by choosing a region $S$ whose boundary can be decomposed into 2 spacelike hypersurfaces $\Sigma_1, \Sigma_2$ (with $\Sigma_2$ being in the future light cone of $\Sigma_1$) joined by a timelike hypersurface $T$ (think a 4d cylinder with axis along the time direction), and taking the orientation of the normal to $\Sigma_1$ to be towards the future direction (thus the opposite orientation to the one it has as a piece of $\partial S$), we get:
$$
0 = \int_{\partial S} J^\mu n_\mu \mathrm{d}\mathbf{\sigma} = - \int_{\Sigma_1} J^\mu n_\mu \mathrm{d}\mathbf{\sigma} + \int_{\Sigma_2} J^\mu n_\mu \mathrm{d}\mathbf{\sigma} + \int_{T} J^\mu n_\mu \mathrm{d}\mathbf{\sigma}
$$
Thus, calling $Q_\Sigma$ the flux of $J^\mu$ through the spacelike surface $\Sigma$:
$$ Q_{\Sigma_2} - Q_{\Sigma_1} = \int_{T} J^\mu n_\mu \mathrm{d}\mathbf{\sigma} $$
Then, if $J^\mu$ (or at least its flux) happens to be null on the timelike piece $T$ of the boundary, $ Q_{\Sigma_2} = Q_{\Sigma_1} = Q$, and if we "glue together" more regions such that the past spacelike boundary of each is the future spacelike boundary of the previous one (and such that $J_\mu$ has zero flux on the timelike boundary of each), $Q_{\Sigma_i} = Q$ for all $i$. This is the conserved charge associated to the current $J^\mu$, and having obtained it in a coordinate-free way it's clearly invariant under coordinate transformations.
Computing $Q_\Sigma$ on a spacelike slice $\Sigma_t$ with constant time coordinate $x_0 = t$ gives the more familiar formula in your question, and if we have coordinates covering the whole spacetime we can take the family of surfaces $\{\Sigma_t\}$ and consider them connected by pieces of timelike surfaces "at spatial infinity" (where the current is usually taken to vanish), thus indeed for the charges $Q_t$ associated to this family of surfaces, $\frac{\mathrm{d} Q_t}{\mathrm{d} t} = 0$ holds.