Dummy variables in Dyson series In the Dyson series, it is known that:
\begin{align}
{\cal T}\exp\left[-\frac{i}{\hbar}\int_0^tH(t')dt'\right] 
&=
I - \frac{i}{\hbar} \int_{0}^{t} dt' H(t')   + \left(-\frac{i}{\hbar}\right)^2 \frac{1}{2} \mathcal{T}\left(\int_{0}^{t} dt' H(t')\right)^2
\\& \quad +
\left(-\frac{i}{\hbar}\right)^3 \mathcal{T}\left(\frac{1}{3!}\left(\int_{0}^{t} dt' H(t')\right)^3\right) \cdots
\\ & =
I - \frac{i}{\hbar}\int_{0}^{t}dt' H(t') + \left(-\frac{i}{\hbar}\right)^2 \int_{0}^{t}dt' \int_{0}^{t'}dt'' H(t') H(t'')
\\ & \quad
+ \left(-\frac{i}{\hbar}\right)^3 \int_{0}^{t}dt' \int_{0}^{t'}dt'' \int_{0}^{t''}dt''' H(t') H(t'') H(t''') +\cdots
\end{align}
I know that by showing the "square" integration is equal to two "triangle" integrals, we have
$$\int_{0}^{t}dt'\int_{0}^{t'}dt'' H(t')H(t'') = \frac{1}{2}\mathcal{T}\left\{\left(\int_{0}^{t}dt'H(t')\right)^2\right\}.$$
I know the R.H.S is the integral of square. But why is $\mathcal{T}$
    needed in$$
\mathcal{T}\left\{\left(\int_{0}^{t}dt'H(t')\right)^2\right\}~?$$
Since $$\left(\int_{0}^{t}dt'H(t')\right)^2 = 
\left(\int_{0}^{t}dt'H(t')\right)\left(\int_{0}^{t}dt'H(t')\right),$$
even by changing the dummy variable
$$\left(\int_{0}^{t}dt'H(t')\right)^2 = 
\left(\int_{0}^{t}dt'H(t')\right)\left(\int_{0}^{t}dt''H(t'')\right),$$
    the two single integral terms in the product mean the same thing. Without messing around this product form into a double integral form, why $\mathcal{T}$ is needed in
    this particular term even before changing of dummy variable?
 A: Time-ordering is needed if the Hamiltonians $H(t^{\prime})$ and $H(t^{\prime\prime})$ at different times do not commute. 
Example: If the Hamiltonian is 
$$H(t) ~=~ \left\{\begin{array}{ccl} \color{Red}{\diamondsuit}&\text{if}& t<0, \cr
\clubsuit &\text{if}& t>0,  \end{array} \right.$$
then the non-time-ordered product $(\int\! dt ~H(t))^2$ has all $2\times2=4$ possible terms: $$\clubsuit\clubsuit , \quad  \color{Red}{\diamondsuit}\color{Red}{\diamondsuit}, \quad  \clubsuit\color{Red}{\diamondsuit}, \quad   \text{and}  \quad  \color{Red}{\diamondsuit}\clubsuit.$$ In contrast, the time-ordered product $T\left[(\int\! dt~ H(t))^2\right]$ does not contain terms with anti-chronological ordering $\color{Red}{\diamondsuit}\clubsuit$.
See also this related Phys.SE post for the necessity to introduce time-ordering.
A: Consider 
$$\int_0^t dt'\int_0^{t}dt'' \mathcal{T}\left(H(t')H(t'')\right)$$
\begin{align}
&=
\int_0^t dt'\int_0^{t'}dt'' \mathcal{T}\left(H(t')H(t'')\right)
\\ & \qquad
+\int_0^t dt'\int_{t'}^{t}dt'' \mathcal{T}\left(H(t')H(t'')\right)
\\&=
\int_0^t dt'\int_0^{t'}dt'' H(t')H(t'')
\\ & \qquad
+\int_0^t dt'\int_{t'}^{t}dt''H(t'')H(t')
\end{align}
The last line follows from the definition of time ordering: in the first integral $t''<t'$ checking the limits of integration, so $\mathcal{T}\left(H(t')H(t'')\right)=H(t')H(t'')$, similarly for the second integral.
As you mentioned using the trick with triangles and squares, we can change the second integral to 
$$\int_0^t dt''\int_{0}^{t''}dt'H(t'')H(t')$$
This is the upper triangle in a $(t',t'')$ diagram. Now relabel dummy variables
$$\int_0^t dt'\int_{0}^{t'}dt''H(t')H(t'')$$
And so recombining we get 
$$\int_0^t dt'\int_0^{t}dt'' \mathcal{T}\left(H(t')H(t'')\right)=2\int_0^t dt'\int_{0}^{t'}dt''H(t')H(t'').$$

Let's take a reasonably straightforward example:
$$H(t)=t\sigma_y +(1-t)\sigma_x$$ 
$H(0)$ doesn't commute with $H(1)$.
Let $$I(t)=\int_0^t dt' H(t')=\frac{t^2}{2}\sigma_y +t(1-\frac{t}{2})\sigma_x$$
Then \begin{align}I(t)^2 &=\int_0^t dt_1 \int_0^t dt_2 H(t_1) H(t_2)\\ &=(\frac{t^4}{4}+t^2(1-t/2)^2)I+\frac{t^3}{2}(1-t/2)\{\sigma_y,\sigma_x\}=t^2(\frac{t^2}{2}-t+1)I \end{align}
Now with time ordering \begin{align} J(t)=\int_0^t dt_1 \int_0^t dt_2 \mathcal{T}\left(H(t_1) H(t_2)\right)&=\int_0^t dt_1\int_0^{t_1}dt_2 H(t_1)H(t_2)+\int_0^t dt_1\int_{t_1}^{t}dt_2 H(t_2)H(t_1)\\ &=\int_0^t dt_1 H(t_1)\int_0^{t_1}dt_2 H(t_2)+\int_0^t dt_1\int_{t_1}^{t}dt_2 H(t_2)H(t_1)\\ &= \int_0^t dt_1 H(t_1)I(t_1)+\int_0^t dt_1 \left(I(t)-I(t_1)\right)H(t_1)\end{align}
It's useful to commute some commutators:
\begin{align}\left[H(t_1),H(t_2)\right]&=\left[ t_1\sigma_y +(1-t_1)\sigma_x , t_2\sigma_y +(1-t_2)\sigma_x \right]\\ &=t_1(1-t_2)[\sigma_y,\sigma_x]+t_2(1-t_1)[\sigma_x,\sigma_y]\\&=2i(t_2-t_1)\sigma_z\end{align}
And 
\begin{align}
\left[H(t_1),I(t_1)\right] &=\left[ H(t_1) \,, \int_0^{t_1} dt_2 H(t_2)\right]\\&= \int_0^{t_1} dt_2 \left[ H(t_1) \,,H(t_2)\right]\\&=-it_1^2\sigma_z
\end{align}
So 
\begin{align}
J(t)&=\int_0^t dt_1 \left[H(t_1),I(t_1)\right] +I(t)^2\\&=-i\frac{t^3}{3}\sigma_z+I(t)^2
\end{align}
Finally lets consider 
$$K(t)=2\int_0^t dt_1 \int_0^{t_1} dt_2 H(t_1)H(t_2) \overset{?}{=} J(t)$$
Well
\begin{align}
K(t)=2\int_0^t dt_1 H(t_1)I(t_1)&=\int_0^t dt_1 \left[H(t_1),I(t_1)\right]+\{H(t_1),I(t_1)\}\\&=-i\frac{t^3}{3}\sigma_z + \int_0^t dt_1 \{H(t_1),I(t_1)\}
\end{align}
Now 
\begin{align}
\{H(t_1),I(t_1)\}=\int_0^{t_1} dt_2 \{H(t_1),H(t_2)\}&=\int_0^{t_1} dt_2 \left(2t_1 t_2 +2(1-t_1)(1-t_2)\right)I\\&=I\left(t_1^3 +2(1-t_1)(1-\frac{t_1}{2})t_1\right)
\end{align}
Finally if you multiply out and integrate you will find that 
$$K(t)=-i\frac{t^3}{6}\sigma_z +t^2(\frac{t^2}{2}-t+1)I=-i\frac{t^3}{6}\sigma_z+I(t)^2=J(t)$$

As an extra note, the definition I've seen for the time ordered exponential is (with the convention that the first term is the identity operator):
$$\mathcal{T}\exp\left(-i/\hbar \int_0^t h(t')dt'\right):=\sum_{n=0}^\infty \frac{1}{n!}\left(\frac{-i}{\hbar}\right)^n \int_0^t dt_1 \ldots\int_0^t dt_n \mathcal{T}\left(H(t_1)\ldots H(t_n)\right) $$
And for the Dyson series:
$$U^\dagger(t,0)=I+\frac{-i}{\hbar}\int_0^t dt' H(t')+\left(\frac{-i}{\hbar}\right)^2\int_0^t dt'\int_0^{t'}dt'' H(t')H(t'')+\ldots$$
The above reasoning at least formally suggests them to be equivalent.
A: You have to keep in mind that an integral is (more or less) a sum, and therefore in a double or multiple integral, you have cross terms that have to be correctly time-ordered. The usual Riemann integral does not take in account the non-commuting nature of its integrand, so you have to make it manifest with the time-ordering symbol $\mathcal{T}$.
Imagine for a moment that you discretize your integral. Without the time-ordering symbol $\mathcal{T}$, you would get
$$ \left(\int_0^t dt' H(t') \right)^2 = \int_0^t dt' \int_0^t dt'' H(t') H(t'') \to \sum_{j=0}^{t/\Delta t} \sum_{k=0}^{t/\Delta t} (\Delta t)^2 H(j\Delta t) H(k\Delta t)$$
Here, all the $j$ ($t'$ in the continuous version) terms come before the $k$ ($t''$), regardless if $t' > t''$.
On the other hand, the same integral with the time-ordering symbol would be
\begin{align}
\mathcal{T}\left(\int_0^t dt' H(t') \right)^2 
& = 
\mathcal{T}\left[\int_0^t dt' \int_0^t dt'' H(t') H(t'')\right] 
\\ & \to \sum_{j=0}^{t/\Delta t} \sum_{k=0}^{t/\Delta t} (\Delta t)^2 \mathcal{T} [H(j\Delta t) H(k\Delta t)]
\\ & =
\sum_{j=0}^{t/\Delta t} \sum_{k=0}^{j} (\Delta t)^2 H(j\Delta t) H(k\Delta t) 
\\ & \qquad + 
\sum_{j=0}^{t/\Delta t} \sum_{k=j}^{t/\Delta t} (\Delta t)^2 H(k\Delta t) H(j\Delta t)
\end{align}
Now, the two Hamiltonians are correctly time-ordered.
The $\mathcal{T}$ in your case comes from the series expansion of the time-ordered exponential $$\mathcal{T}\exp\left(-\frac{i}{\hbar}\int_0^t dt' H(t)\right).$$
tl;dr: The $\mathcal{T}$ determines the prescription for the Riemann sums when you're integrating non-commuting objects.
