You can take the expression $C=\frac{\delta Q}{\mathrm dT}$ as the infinitesimal version of
$$
C=\frac{Q}{\Delta T}
$$
or a formal rewrite of
$$
\delta Q=C\mathrm dT
$$
which, however, doesn't make sense in the language of differential forms as division by the form $\mathrm dT$ is not defined.
Let's take a look at the meaning of $\delta Q=C\mathrm dT$ assuming differential forms:
By the second law of thermodynamics, $\delta Q = T\mathrm dS$. The $\delta$ has no special meaning, it's just a reminder that we're dealing with a differential form and not a function (we can't write $\mathrm dQ$ here as the form is not exact, ie not the differential of some state function $Q$).
Thermodynamical systems are in general at least two-dimensional and allow different choices of coordinates, so assume $S$ is represented by a function of temperature and another variable, eg $S=S(V,T)$ or $S=S(P,T)$.
The definition of heat capacity from above assumes that $S$ is a function of $T$ alone as the right-hand side doesn't contain terms with $\mathrm dV$ or $\mathrm dP$. In general, we thus need a further restriction on permitted processes, like $V=\mathrm{const}$ or $P=\mathrm{const}$, which yields $C_V$ or $C_P$ respectively.
Under this assumption, we have
$$
\mathrm dS = \frac{\partial S}{\partial T} \mathrm dT
$$
ie
$$
C\mathrm dT = \delta Q = T\frac{\partial S}{\partial T} \mathrm dT
$$
and finally
$$
C = T\frac{\partial S}{\partial T}
$$
A further note for the more mathematically inclined:
Geometrically, the restrictions $V=\mathrm{const}$ or $P=\mathrm{const}$ define a 1-dimensional submanifold where the pullback of $\delta Q$ via the natural embedding will be (locally) exact. In fact, this pullback needs to be included to make the equations above conform to the notation used in differential geometry:
Let $\nu$ be our embedding with $\mathrm d\tau = \nu^*\mathrm dT$ non-degenerate. There's a function $C_\nu$ and (as $\nu^*\delta Q$ is closed) another function $Q_\nu$ (or rather a family of locally defined functions) with
$$
\nu^*\delta Q = C_\nu \mathrm d\tau = \mathrm dQ_\nu
$$
that is
$$
C_\nu = \frac{\partial Q_\nu}{\partial\tau}
$$
In case of $V=\mathrm{const}$, $Q_\nu$ is the pullback of the internal energy $U$, whereas in case of $P=\mathrm{const}$, $Q_\nu$ is the pullback of the Enthalpy $H$.
In physicist's notation this reads
$$
C_V = \left(\frac{\partial U}{\partial T}\right)_V \\
C_P = \left(\frac{\partial H}{\partial T}\right)_P
$$