The standard normalization of Gell-Mann matrices (in the fundamental, triplet, of su(3)) is
$$
\operatorname{Tr} (\lambda_a\lambda_b) = 2\delta_{ab},
$$
and all 8 matrices are traceless, i.e. orthogonal to the 3-identity.
But note squares of generators are not in the Lie algebra, but in the universal enveloping algebra, and their properties vary with representation! Further note the "miracle"
$$
\sigma_1^2=\sigma_2^2=\sigma_3^2=1\!\!1_2
$$
is strictly a feature of isospin in the doublet, and so 1/3 the value of the respective su(2) Casimir invariant, embedded in the same subspace of su(3). Check this fails for the triplet representation of isospin (in the octet of su(3))!
You are then only talking about diagonal matrices
$\lambda_3, \lambda_3^2, \lambda_8, \lambda_8^2, 1\!\!1_3$,
(where $\lambda_1^2=\lambda_2^2~(=\lambda_3^2)$ come along for the ride and need not be discussed). You are discussing
three mutually orthogonal 3-vectors, $V(\lambda_3),V(\lambda_8), 1\!\!1$ and fussing their normalizations following from above:
$$
V(\lambda_3)\cdot V( 1\!\!1)=0,\\
V(\lambda_8)\cdot V( 1\!\!1)=0,\\
V(\lambda_8)\cdot V(\lambda_3)=0 \\
V(\lambda_3^2)\cdot V(\lambda_3)=0,\\
V(\lambda_8)\cdot V(\lambda_3^2)=0 \\
V(\lambda_8^2)\cdot V(\lambda_8^2)=2 \\
V(\lambda_3^2)\cdot V(\lambda_3^2)=2 .
$$
It is thus necessary that the two vectors $V(\lambda_3^2)= (1,1,0)^T$ and $V(\lambda_8^2)=(1,1,4)^T/3$ be expressible as linear combinations of the fundamental three such above,
actually two, being orthogonal to $V(\lambda_3)$,
$$
V(\lambda_3^2) ={2\over 3} 1\!\! 1 +{1\over \sqrt{3} }\lambda_8, \\
V(\lambda_8^2) = {2\over 3} 1\!\! 1 -{1\over \sqrt{3}} \lambda_8,
$$
as you confirmed. The 2/3 coefficient of the identity is to align its trace/normalization, 3, with that of the λ matrices. I'm not sure why this should be bizarre.