The formal solution of the time-dependent Schrödinger equation Consider the time-dependent Schrödinger equation (or some equation in Schrödinger form) written down as
$$
\tag 1 i\hbar \partial_{t} \Psi ~=~ \hat{H} \Psi .
$$
Usually, one likes to write that it has a formal solution of the form
$$
\tag 2 \Psi (t) ~=~ \exp\left[-\frac{i}{\hbar} \int \limits_{0}^{t} \hat{ H}(t^{\prime}) ~\mathrm dt^{\prime}\right]\Psi (0).
$$
However, this form for the solution of $(1)$ is actually built by the method of successive approximations which actually returns a solution of the form
$$
\tag 3 \Psi (t) ~=~ \color{red}{\hat{\mathrm T}} \exp\left[-\frac{i}{\hbar} \int \limits_{0}^{t} \hat{H}(t^{\prime})~\mathrm  dt^{\prime}\right]\Psi (0), \qquad t>0,
$$
where $\color{red}{\hat{\mathrm T}}$ is the time-ordering operator.
It seems that $(3)$ doesn't coincide with $(2)$, but formally $(2)$ seems to be perfectly fine: it satisfies $(1)$ and the initial conditions. So where is the mistake?
 A: The existing answer by Qmechanic is entirely correct and extremely thorough. But it is very long and technical, and there's a danger that the core of the answer can get buried under all of that.
The claim made in the question,

formally $$
\tag 2 \Psi (t) ~=~ \exp\left[-i \int_{0}^{t} \hat{ H}(t^{\prime}) ~\mathrm dt^{\prime}\right]\Psi (0)
$$
seems to be perfectly fine: it satisfies
$$
\tag 1 i \partial_{0} \Psi ~=~ \hat{ H}~ \Psi 
$$
and the initial conditions

is incorrect: the wavefunction in $(2)$ does not satisfy the differential equation in $(1)$.
The reason for this is that, as a rule, the exponential of an operator $\hat A(t)$ does not obey the differential equation
$$
\frac{\mathrm d}{\mathrm dt}e^{\hat A(t)} \stackrel{?}{=} \frac{\mathrm d \hat{A}}{\mathrm dt} e^{\hat A(t)}
$$
that one might naively hope it to satisfy. (It is, after all, the exponential operator, right?)
To see why this doesn't work, consider the series expansion of the exponential:
\begin{align}
\frac{\mathrm d}{\mathrm dt}e^{\hat A(t)}
& = \frac{\mathrm d}{\mathrm dt}\sum_{n=0}^\infty \frac{1}{n!}\hat A(t)^n
\\& = \sum_{n=0}^\infty \frac{1}{n!} \frac{\mathrm d}{\mathrm dt} \hat A(t)^n,
\end{align}
and so far so good. However, if we try to push this further, we don't get a nice derivative of the form $\frac{\mathrm d}{\mathrm dt} \hat A(t)^n \stackrel{?}{=} n\frac{\mathrm d\hat A}{\mathrm dt} \hat A(t)^{n-1}$ like we do for scalar-valued functions.
Instead, when we apply the product rule, we get the individual derivatives of each of the operators in the product, at their place within the product:
$$
\frac{\mathrm d}{\mathrm dt} \hat A(t)^n
=
\frac{\mathrm d\hat A}{\mathrm dt} \hat A(t)^{n-1}
+\hat A(t)\frac{\mathrm d\hat A}{\mathrm dt} \hat A(t)^{n-2}
+\hat A(t)^2\frac{\mathrm d\hat A}{\mathrm dt} \hat A(t)^{n-3}
+\cdots
+\hat A(t)^{n-2}\frac{\mathrm d\hat A}{\mathrm dt} \hat A(t)
+\hat A(t)^{n-1}\frac{\mathrm d\hat A}{\mathrm dt} 
.
$$
This can simplify to just $n\frac{\mathrm d\hat A}{\mathrm dt} \hat A(t)^{n-1}$ but only under the condition that $\hat A(t)$ commute with its derivative,
$$
\left[\frac{\mathrm d\hat A}{\mathrm dt} , \hat A(t)\right] \stackrel{?}{=} 0,
$$
and as a general rule this is not satisfied.
For the particular case in the question, where $\hat A(t) = -i \int_0^t \hat H(\tau) \mathrm d\tau$ and therefore $\frac{\mathrm d\hat A}{\mathrm dt} = -i \hat H(t)$, we have, by linearity,
$$
\left[\frac{\mathrm d\hat A}{\mathrm dt} , \hat A(t)\right] 
=
 -i \int_0^t \left[\hat H(t),\hat H(\tau)\right] \mathrm d\tau,
$$
so if there are any times $t'<t$ for which $\left[\hat H(t),\hat H(t')\right]$ is not zero, then the whole house of cards comes down.
A: I) The solution to the time-dependent Schrödinger equation (TDSE) is
$$ \Psi(t_2) ~=~ U(t_2,t_1) \Psi(t_1),\tag{A}$$
where the (anti)time-ordered exponentiated Hamiltonian
$$\begin{align} U(t_2,t_1)~&=~\left\{\begin{array}{rcl}
T\exp\left[-\frac{i}{\hbar}\int_{t_1}^{t_2}\! dt~H(t)\right]
&\text{for}& t_1 ~<~t_2 \cr\cr
AT\exp\left[-\frac{i}{\hbar}\int_{t_1}^{t_2}\! dt~H(t)\right]
&\text{for}& t_2 ~<~t_1 \end{array}\right.\cr\cr
~&=~
\underset{N\to\infty}{\lim}
\exp\left[-\frac{i}{\hbar}H(t_2)\frac{t_2-t_1}{N}\right] \cdots\exp\left[-\frac{i}{\hbar}H(t_1)\frac{t_2-t_1}{N}\right]\end{align}\tag{B} $$
is formally the unitary evolution operator, which satisfies its own two TDSEs
$$ i\hbar \frac{\partial }{\partial t_2}U(t_2,t_1)
~=~H(t_2)U(t_2,t_1),\tag{C} $$
$$i\hbar \frac{\partial }{\partial t_1}U(t_2,t_1)
~=~-U(t_2,t_1)H(t_1),\tag{D} $$
along with the boundary condition
$$ U(t,t)~=~{\bf 1}.\tag{E}$$
II) The evolution operator $U(t_2,t_1)$ has the group-property
$$  U(t_3,t_1)~=~U(t_3,t_2)U(t_2,t_1). \tag{F}$$
The (anti)time-ordering in formula (B) is instrumental for the (anti)time-ordered expontial (B) to factorize according to the group-property (F).
III) The group property (F) plays an important role in the proof that formula (B) is a solution to the TDSE (C):
$$\begin{array}{ccc}  \frac{U(t_2+\delta t,t_1) -  U(t_2,t_1)}{\delta t} &\stackrel{(F)}{=}&
\frac{U(t_2+\delta t,t_2) - {\bf 1} }{\delta t}U(t_2,t_1)\cr\cr
\downarrow & &\downarrow\cr\cr 
\frac{\partial }{\partial t_2}U(t_2,t_1)
&& -\frac{i}{\hbar}H(t_2)U(t_2,t_1).\end{array}\tag{G}$$
Remark: Often the (anti)time-ordered exponential formula (B) does not make mathematical sense directly. In such cases, the TDSEs (C) and (D) along with boundary condition (E) should be viewed as the indirect/descriptive defining properties of the (anti)time-ordered exponential (B).
IV) If we define the unitary operator without the (anti)time-ordering in formula (B) as
$$ V(t_2,t_1)~=~\exp\left[-\frac{i}{\hbar}\int_{t_1}^{t_2}\! dt~H(t)\right],\tag{H}$$
then the factorization (F) will in general not take place,
$$  V(t_3,t_1)~\neq~V(t_3,t_2)V(t_2,t_1). \tag{I}$$
There will in general appear extra contributions, cf. the BCH formula. Moreover, the unitary operator $V(t_2,t_1)$ will in general not satisfy the TDSEs (C) and (D). See also the example in section VII.
V) In the special (but common) case where the Hamiltonian $H$ does not depend explicitly on time, the time-ordering may be dropped. Then formulas (B) and (H) reduce to the same expression
$$ U(t_2,t_1)~=~\exp\left[-\frac{i}{\hbar}\Delta t~H\right]~=~V(t_2,t_1), \qquad \Delta t ~:=~t_2-t_1.\tag{J}$$
VI) Emilio Pisanty advocates in a comment that it is interesting to differentiate eq. (H) w.r.t. $t_2$ directly. If we Taylor expand the exponential (H) to second order, we get
$$ \frac{\partial V(t_2,t_1)}{\partial t_2}
~=~-\frac{i}{\hbar}H(t_2) -\frac{1}{2\hbar^2} \left\{ H(t_2), \int_{t_1}^{t_2}\! dt~H(t) \right\}_{+} +\ldots,\tag{K} $$
where $\{ \cdot, \cdot\}_{+}$ denotes the anti-commutator. The problem is that we would like to have the operator $H(t_2)$ ordered to the left [in order to compare with the TDSE (C)]. But resolving the anti-commutator may in general produce un-wanted terms. Intuitively without the (anti)time-ordering in the exponential (H), the $t_2$-dependence is scattered all over the place, so when we differentiate w.r.t. $t_2$, we need afterwards to rearrange all the various contributions to the left, and that process generate non-zero terms that spoil the possibility to satisfy the TDSE (C). See also the example in section VII.
VII) Example. Let the Hamiltonian be just an external time-dependent source term
$$ H(t) ~=~ \overline{f(t)}a+f(t)a^{\dagger}, \qquad [a,a^{\dagger}]~=~\hbar{\bf 1},\tag{L}$$
where $f:\mathbb{R}\to\mathbb{C}$ is a function. Then according to Wick's Theorem
$$ T[H(t)H(t^{\prime})] ~=~ : H(t) H(t^{\prime}): ~+ ~C(t,t^{\prime}), \tag{M}$$
where the so-called contraction
$$ C(t,t^{\prime})~=~ \hbar\left(\theta(t-t^{\prime})\overline{f(t)}f(t^{\prime})
+\theta(t^{\prime}-t)\overline{f(t^{\prime})}f(t)\right) ~{\bf 1}\tag{N}$$
is a central element proportional to the identity operator. For more on Wick-type theorems, see also e.g. this, this, and this Phys.SE posts. (Let us for notational convenience assume that $t_1<t_2$ in the remainder of this answer.) Let
$$  A(t_2,t_1)~=~-\frac{i}{\hbar}\int_{t_1}^{t_2}\! dt~H(t)
~=~-\frac{i}{\hbar}\overline{F(t_2,t_1)} a -\frac{i}{\hbar}F(t_2,t_1) a^{\dagger} ,\tag{O}$$
where
$$ F(t_2,t_1)~=~\int_{t_1}^{t_2}\! dt ~f(t). \tag{P}$$
Note that
$$
\frac{\partial }{\partial t_2}A(t_2,t_1)~=~-\frac{i}{\hbar}H(t_2), \qquad
\frac{\partial }{\partial t_1}A(t_2,t_1)~=~\frac{i}{\hbar}H(t_1).\tag{Q} $$
Then the unitary operator (H) without (anti)time-order reads
$$\begin{align}
V(t_2,t_1)~&=~e^{A(t_2,t_1)}
\\
~&=~\exp\left[-\frac{i}{\hbar}F(t_2,t_1) a^{\dagger}\right]\exp\left[\frac{-1}{2\hbar}|F(t_2,t_1)|^2\right]\exp\left[-\frac{i}{\hbar}\overline{F(t_2,t_1)} a\right].\tag{R}
\end{align}$$
Here the last expression in (R) displays the normal-ordered for  of $V(t_2,t_1)$. It is a straightforward exercise to show that formula (R) does not satisfy TDSEs (C) and (D). Instead the correct unitary evolution operator is
$$\begin{align}
U(t_2,t_1)~&\stackrel{(B)}{=}~T\exp\left[-\frac{i}{\hbar}\int_{t_1}^{t_2}\! dt~H(t)\right]
\\~&\stackrel{(M)}{=}~:\exp\left[-\frac{i}{\hbar}\int_{t_1}^{t_2}\! dt~H(t)\right]:~ \exp\left[\frac{-1}{2\hbar^2}\iint_{[t_1,t_2]^2}\! dt~dt^{\prime}~C(t,t^{\prime})\right]
\\ ~&=~ e^{A(t_2,t_1)+D(t_2,t_1)}~=~V(t_2,t_1)e^{D(t_2,t_1)}\tag{S}, 
\end{align}$$
where
$$ D(t_2,t_1)~=~\frac{{\bf 1}}{2\hbar}\iint_{[t_1,t_2]^2}\! dt~dt^{\prime}~{\rm sgn}(t^{\prime}-t)\overline{f(t)}f(t^{\prime})\tag{T}$$
is a central element proportional to the identity operator. Note that
$$\begin{align}
\frac{\partial }{\partial t_2}D(t_2,t_1)~&=~\frac{{\bf 1}}{2\hbar}\left(\overline{F(t_2,t_1)}f(t_f)-\overline{f(t_2)}F(t_2,t_1)\right)
\\ ~&=~\frac{1}{2}\left[ A(t_2,t_1), \frac{i}{\hbar}H(t_2)\right]~=~\frac{1}{2}\left[\frac{\partial }{\partial t_2}A(t_2,t_1), A(t_2,t_1)\right].\tag{U}
\end{align}$$
One may use identity (U) to check directly that the operator (S) satisfy the TDSE (C).
References:

*

*Sidney Coleman, QFT lecture notes, arXiv:1110.5013; p. 77.

A: The equation
$$\partial _{t}\psi (t)=-iH\psi (t)$$
acting in a Hilbert space with $H$ self-adjoint has the general solution
$$\psi (t)=\exp [-iH(t-t_{0})]\psi (t_{0}),$$
by Stone's theorem. In case $H=H(t)$ depends on $t$ matters change and time
ordering becomes relevant. If $H$ does not depend on time your Eq. (3)
reduces to (2).
