So a little tinkering shows the eigenvalues are given by the sequence
$$
\lambda_n=\left\{\begin{array}{ll}\frac{(-1)^n}{4^n} {{2n}\choose{n}}&\qquad n=0,1,\ldots,k, \\
0& \qquad n> k\, ,\end{array}\right.\tag{1}
$$
where $k$ is the integer part of $j$.
You are correct that $1$ is an eigenvalue. In fact, your matrix $A$ is a non-negative matrix and there are lots of results on this. In particular the largest eigenvalues is the largest of the sums of rows or columns, which here is always $1$. You can check out some of the theory of non-negative matrices in
Matrix analysis, Horn&Johnson, Cambridge University Press,
The theory of matrices, F. Gantmacher, published by the AMS.
I was unable guess the sequence from first principle, so I had to generate the matrices and obtain the eigenvalues using Mathematica to then recover the sequence manually. Mathematica returns an expression for the non-zero eigenvalues in term of the Pochhammer symbol which simplifies to
\begin{align}
(-1)^n\frac{\Gamma \left(n+\frac{1}{2}\right)}{\sqrt{\pi } \,\Gamma (n+1)}\, .
\end{align}
You can then get rid of the $\Gamma(n+\frac{1}{2})$ by using the duplication formula
\begin{align}
\Gamma(2n)=\frac{2^{2n-1}\Gamma(n)\Gamma(n+\frac{1}{2})}{\sqrt{\pi}}
\end{align}
Possibly the most convenient form the entries for your $A$ matrix is not to start with $d^j$ in terms of CGs but rather uses the expression
\begin{align}
d^j_{mm'}(\pi/2)&=(-1)^{m-m'}\frac{1}{2^j}
\sqrt{\frac{(j+m)!(j-m)!}{(j+m')!(j-m')!}}
\sum_k (-1)^k {{j+m'}\choose{k}} {{j-m'}\choose{k+m-m'}}\, .
\end{align}
There is no easy way to sum this into something simpler to manipulate, or get insight into the the actual entries, so it's hard to understand the structure of the eigenvalues unless one computes them.
As an example, for $j=5/2$, your matrix $A$ has the form
\begin{align}
A=\frac{1}{32}\left(\begin{array}{cccccc}
1 & 5 & 10 & 10 & 5 & 1 \\
5 & 9 & 2 & 2 & 9 & 5 \\
10 & 2 & 4 & 4 & 2 & 10 \\
10 & 2 & 4 & 4 & 2 & 10 \\
5 & 9 & 2 & 2 & 9 & 5 \\
1 & 5 & 10 & 10 & 5 & 1 \\
\end{array}
\right)\, .
\end{align}
The bi-symmetric nature of the matrix is obvious, but there's no obvious pattern in the entries by just staring at them. The eigenvalues are correctly given by (1) as $\{1,-\frac{1}{2},\frac{3}{8},0,0,0\}$.
BTW there's a recursive structure to the first and last row and the first and last column of the matrix. Consider the "immediately preceeding" case where $j=2$. Then your matrix has the form
\begin{align}
B=\frac{1}{16}\left(
\begin{array}{ccccc}
1 & 4 & 6 & 4 & 1 \\
4 & 4 & 0 & 4 & 4 \\
6 & 0 & 4 & 0 & 6 \\
4 & 4 & 0 & 4 & 4 \\
1 & 4 & 6 & 4 & 1 \\
\end{array}
\right)
\end{align}
Comparing with the $j=5/2$ case, the common factor in front is multiplied by $1/2$, and the entry $a_{1j}$ for $j=2,3,4,5$ on first row of for the $j=5/2$ case is the sum $b_{1,j-1}+b_{1,j}$ of the $j=2$ case. Thus, $a_{12}=b_{11}+b_{12}$, $a_{13}=b_{12}+b_{23}$ etc. and likewise for the last row, first column and last column. This fails for other rows. There might something there to explain why the non-zero eigenvalues for $j=2$ are a subset of those for $j=5/2$...
