I think I figured out a simple (algebraic) way of proving the statement. For simplicity, let $S = \exp{(i\pi J_2)}$. Using the Adjoint/adjoint representation, it's clear that
$$
SJ_3 S^{-1} = -J_3
$$
and that
$$
S J_+ S^{-1}=-J_-
$$
Notice that
\begin{align}
SJ_3 S^{-1} S|jm\rangle &= SJ_3 |jm\rangle \\
-J_3 S|jm\rangle &= mS |jm\rangle \\
J_3 S|jm\rangle &= -mS |jm\rangle
\end{align}
Therefore, $S|jm\rangle$ is an eigenvector of $J_3$ with eigenvalue $-m$ and thus $S|jm\rangle = c_m |j,-m\rangle$ where $|c_m| = 1$ since $S$ is unitary. Apply a similar argument with $J_+,J_-$, i.e.,
\begin{align}
SJ_+ S^{-1} S|j,m\rangle &= SJ_+ |j,m\rangle \\
-J_- S|j,m\rangle &= S \sqrt{j(j+1)-m(m+1)}|j,m+1\rangle \\
-c_m \sqrt{j(j+1)-m(m+1)} |j,-m-1\rangle &= c_{m+1} \sqrt{j(j+1)-m(m+1)}|j,-m-1\rangle \\
\end{align}
Hence, if $m<j$, then $c_{m}=-c_{m+1}$. In particular, $c_{j-m}=(-1)^m c$ where I set $c=c_j$. Therefore, the key now is to show what $c$ is.
Notice that $S^4=1$. Hence, $c^4=1$ and thus $c=\pm 1, \pm i$. Notice that $S$ is real with respect to the orthonormal basis $|j,m\rangle$. Hence, $c=\pm 1$. However, I cannot tell whether $c=1$ or $=-1$.
EDIT: I realized that my original final step wasn't enough to prove the statement, so I changed it a little, but I'm still stuck between $\pm 1$.
EDIT2: Notice that if $j=l$ is an integer, then the character of $SO(3)$ is given by
$$
\chi_l (\alpha) = \frac{\sin{((l+1/2)\alpha)}}{\sin{(\alpha/2)}}
$$
where $\alpha$ denotes rotation about any axis of angle $\alpha$. In particular, if $\alpha =\pi$, then the right-hand-side $=(-1)^l$. Notice that the left-hand-side can be given by
$$
\sum_{m=-l}^l \langle lm| e^{i\pi J_2} |lm\rangle = c(-1)^l
$$
since the only nonzero term in the summation is when $m=0$. Hence, in this case, we have $c=+1$.
EDIT 3: (Finally!) I think I have found a proof for the case where $j=s$ is a half-integer. Indeed, if $j=1/2$, it's clear that we should take $c=1$, since
$$
\exp (i\pi J_2) = \exp \left(\frac{\pi}{2} i\sigma_2 \right) = i\sigma_2
$$
where $\sigma_2$ is the Pauli matrix. Now let $j=s$ be a half-integer and $V_j$ denote the corresponding irrep. Then we have
$$
V_j \otimes V_{1/2} = V_{j+1/2} \oplus V_{j-1/2}
$$
More specifically, we should notice that
\begin{align}
\left|j+\frac12,j+\frac12\right\rangle &= |j,j\rangle \left|\frac12,\frac12\right\rangle \\
\left|j+\frac12,-j-\frac12\right\rangle &= |j,-j\rangle \left|\frac12,-\frac12\right\rangle
\end{align}
Indeed, since the basis is unique only up to a global phase, we can always choose $|j,j\rangle \left|1/2,1/2\right\rangle$ to be $|j+1/2,j+1/2\rangle$. By successively applying $J_-$, we see that each time we obtain a nonnegative coefficient in front of the basis and thus we arrive at the second equation, e.g.,
$$
J_- |j,j\rangle \left|\frac12,\frac12\right\rangle = (\cdots) |j,j-1\rangle \left|\frac12,\frac12\right\rangle +(\cdots) |j,j\rangle \left|\frac12,-\frac12\right\rangle
$$
where $(\cdots)$ represents some nonnegative coefficient. Notice that
$$
S |j,j\rangle \left|\frac12,\frac12\right\rangle = c_j |j,-j\rangle \left|\frac12,-\frac12\right\rangle
$$
where $c_j$ is that corresponding to the action of $S$ on $|j,j\rangle$. Notice that $j+1/2$ is an integer and thus $S$ should act on $|j+1/2, j+1/2\rangle$ by $+1$, and thus $c_j=+1$.
