You are correct, and your solutions are wrong. Acting on the basis ket $|10 \rangle$, the right-hand side maps to
\begin{align}
\mathsf{CNOT} \ \mathsf{SWAP}|10\rangle
& =
\mathsf{CNOT}|01\rangle
\\ & = |01\rangle.
\end{align}
Their assertion for the left-hand side is correct, though:
\begin{align}
H^{\otimes 2} \, \mathsf{CNOT} \, H^{\otimes 2}|10\rangle
& =
H^{\otimes 2} \, \mathsf{CNOT} \frac{|0\rangle-|1\rangle}{\sqrt{2}}|+\rangle
\\& =
H^{\otimes 2} \frac{|0\rangle|+\rangle-|1\rangle\,X|+\rangle}{\sqrt{2}}
\\& =
H^{\otimes 2} \frac{|0\rangle|+\rangle-|1\rangle|+\rangle}{\sqrt{2}}
\\& =
H^{\otimes 2} \frac{|0\rangle-|1\rangle}{\sqrt{2}}|+\rangle
\\ & = |10\rangle.
\end{align}
As such, under the standard conventions, the equality is false.
On the other hand, it is true that
$$
H^{\otimes 2} \, \mathsf{CNOT} \, H^{\otimes 2}
=
\mathsf{SWAP} \ \mathsf{CNOT} \ \mathsf{SWAP}.
$$
Either way, you should check with your instructor (or, if self-studying, find a better problem set).