It is straightforward to show that $$ \left[T_\leftarrow\exp\left(-i\int_0^tH(\tau)d\tau\right)\right]^{-1} = T_\rightarrow\exp\left(i\int_0^tH(\tau)d\tau\right), $$ where $T_\leftarrow$ and $T_\rightarrow$ impose time-ordering and anti-time-ordering, respectively.
I'm wondering what the inverse of $$ \left[T_\leftarrow\exp\left(-i\int_0^t\int_0^\tau H(\tau)H(\tau')d\tau\,d\tau'\right)\right]^{-1} = ? $$ might be. Just choosing anti-time-ordering doesn't work. Anti-time-ordering of the "first" time index ($\tau$ in this case) also fails.