Why isn't there a second baryon octet? Let's temporarily ignore spin. If 3 denotes the standard representation of $SU(3)_F$, 1 the trivial rep, 8 the adjoint rep and 10 the symmetric cube, then it's well-known that
$$ 3 \otimes 3 \otimes 3 = 1 \oplus 8 \oplus 8 \oplus 10.$$
Interpreting the 3's as the space of up/down/strange flavour states for a quark, the tensor cube is interpreted as the space of baryon states that can be obtained by combining three light quarks. Obviously spin matters, but at least this should give a classification of baryons modulo spin into $SU(3)_F$-multiplets.
There are two octets here, but in the literature I have only seen one of them described. What is the second octet?
I appreciate that the answer may be "it's more complicated than that".
 A: I think that the answer is that there is a flavor symmetric octet representation and a flavor antisymmetric octet representatio, while the decuplet is totally symmetric. Therefore, when you consider the spin and flavor wavefunction of a baryon for an octet baryon you have: $\chi(spin)\cdot\phi(flavor)=\frac{1}{\sqrt{2}}(\chi^{1/2}_s\cdot 8_s+\chi^{1/2}_a\cdot 8_a)$. where "s" denotes symmetric and "a" antisymmetric. 1/2 superscript denotes a 1/2 spin particle.
A: 

These two Figures are excerpt of a page in : A Modern Introduction to Particle Physics, by Fayyazuddin & Riazuddin, 2nd Edition 2000.
The first shows the well-known octet of the mixed antisymmetric tensor $\:\boldsymbol{8}\:$ while the second shows the octet of the mixed symmetric tensor $\:\boldsymbol{8'}\:$. I don't know what particles, if any, are represented by the latter octet.
See also my answer here : Symmetry in terms of matrices. Therein octet $\:\boldsymbol{8}\:$ is produced by the mixed antisymmetric tensor $\:Y_{ijk}\:$, see equations (B.25) and (B.35), while octet $\:\boldsymbol{8'}\:$ is produced by the mixed symmetric tensor $\:X_{ijk}\:$, see equations (B.24) and (B.37).

The Figures below are excerpts from : A Modern Introduction to Particle Physics-Volume 1: Quantum Field Theory and Particles, by Y.Nagashima, Edition 2010.

 

In 'QUARKS AND LEPTONS: An Introductory Course in Modern Particle Physics', F.Halzen-A.Martin, Edition 1984, we meet the following concerning the spin-up proton: 
We take the mixed-antisymmetric and mixed-symmetric states
\begin{align}   
\mathrm p_{_A} & =\sqrt{\tfrac12}\left(\mathrm u \mathrm d- \mathrm d\mathrm u \right)\mathrm u \:\: \left\{\in \boldsymbol{8}_{_{MA}}\equiv \boldsymbol{8}\right\}
\tag{2.60}\\ 
\mathrm p_{_S} & =\sqrt{\tfrac16}\bigl[\left(\mathrm u \mathrm d+ \mathrm d\mathrm u \right)\mathrm u-2\mathrm u \mathrm u \mathrm d \bigr]\:\: \left\{\in \boldsymbol{8}_{_{MS}}\equiv \boldsymbol{8'}\right\}
\tag{2.62} 
\end{align}
The state $\:\mathrm p_{_A} \:$ is the first member of octet $\:\boldsymbol{8}\:$ shown in the first Figure above while the state $\:\mathrm p_{_S} \:$ is the first member of octet $\:\boldsymbol{8'}\:$ shown in the second Figure.
These are multiplets produced in $\:\rm SU(2)-$isospin and by analogy the $\:\rm SU(2)-$spin antisymmetric, symmetric multiplets are produced by replacing in (2.60), (2.62)
\begin{align}   
\mathrm u  & \quad \Longrightarrow \quad \uparrow
\nonumber\\ 
\mathrm d  & \quad \Longrightarrow \quad \downarrow
\tag{01} 
\end{align}
so 
\begin{align}   
\chi\left(M_A\right) & =\sqrt{\tfrac12}\left(\uparrow \downarrow\uparrow  -\downarrow\uparrow\uparrow  \right)
\nonumber\\ 
\chi\left(M_S\right) & =\sqrt{\tfrac16}\left(\uparrow \downarrow\uparrow  +\downarrow\uparrow\uparrow -2\uparrow\uparrow \downarrow \right)
\tag{2.65} 
\end{align}
Next the spin-up proton is derived from
\begin{equation}   
\vert \mathrm p\!\uparrow\, \rangle=\sqrt{\tfrac12}\bigl[\mathrm p_{_A}\chi\left(M_A\right)+\mathrm p_{_S}\chi\left(M_S\right) \bigr]
\tag{02} 
\end{equation}
where
\begin{align}  
\mathrm p_{_A}\chi\left(M_A\right) & \simeq \left(\mathrm u \mathrm d \mathrm u-\mathrm d \mathrm u \mathrm u\right)\left(\uparrow \downarrow\uparrow  -\downarrow\uparrow\uparrow \right)
\nonumber\\ 
& =\bigl(\mathrm u\!\!\uparrow\!\mathrm d\!\!\downarrow\!\mathrm u\!\!\uparrow -\mathrm u\!\!\downarrow\!\mathrm d\!\!\uparrow\!\mathrm u\!\!\uparrow -\mathrm d\!\!\uparrow\!\mathrm u\!\!\downarrow\!\mathrm u\!\!\uparrow +\mathrm d\!\!\downarrow\!\mathrm u\!\!\uparrow\!\mathrm u\!\!\uparrow  \bigr)
\tag{03} 
\end{align}
and
\begin{align}  
\mathrm p_{_S}\chi\left(M_S\right) & \simeq\left(\mathrm u \mathrm d\mathrm u + \mathrm d\mathrm u \mathrm u-2\mathrm u \mathrm u \mathrm d \right)\left(\uparrow \downarrow\uparrow  +\downarrow\uparrow\uparrow -2\uparrow\uparrow \downarrow \right)
\nonumber\\ 
& =\mathrm u\!\!\uparrow\!\mathrm d\!\!\downarrow\!\mathrm u\!\!\uparrow      + \mathrm u\!\!\downarrow\!\mathrm d\!\!\uparrow\!\mathrm u\!\!\uparrow-2  \mathrm u\!\!\uparrow\!\mathrm d\!\!\uparrow\!\mathrm u\!\!\downarrow +\mathrm d\!\!\uparrow\!\mathrm u\!\!\downarrow\!\mathrm u\!\!\uparrow +\mathrm d\!\!\downarrow\!\mathrm u\!\!\uparrow\!\mathrm u\!\!\uparrow +\cdots
\tag{04} 
\end{align}
and finally

\begin{align}   
\vert \mathrm p\!\uparrow\, \rangle & =\sqrt{\tfrac{1}{18}}\bigl[\mathrm u \mathrm u \mathrm d \left(\uparrow \downarrow\uparrow  +\downarrow\uparrow\uparrow -2\uparrow\uparrow \downarrow \right)+\mathrm u \mathrm d \mathrm u \left(\uparrow\uparrow\downarrow  +\downarrow\uparrow\uparrow -2\uparrow\downarrow \uparrow \right)+\mathrm d \mathrm u \mathrm u \left(\uparrow\downarrow  \uparrow+\uparrow\uparrow\downarrow -2\downarrow \uparrow\uparrow \right)\bigr]
\nonumber\\
&=\sqrt{\tfrac{1}{18}}\bigl[\mathrm u\!\!\uparrow\!\mathrm u\!\!\downarrow\!\mathrm d\!\!\uparrow +\mathrm u\!\!\downarrow\!\mathrm u\!\!\uparrow\!\mathrm d\!\!\uparrow -2\mathrm u\!\!\uparrow\!\mathrm u\!\!\uparrow\!\mathrm d\!\!\downarrow +\text{permutations}\bigr] 
\tag{2.71} 
\end{align}


From above notes we conclude that the real baryon $\:(1/2)^{+}\:$ octet is a combination of the two octets with mixed symmetry $\:\boldsymbol{8},\boldsymbol{8'}$. 
A: This answer is basically equivalent to OP's own answer using slightly different words.

*

*Firstly, we will need a formula
$$\dim (V^{\odot n})~=~\begin{pmatrix} n-1+\dim V \cr n \end{pmatrix} \tag{1}$$
for the dimension of the symmetric tensor product $V^{\odot n}$ of a finite-dimensional vector space $V$. The proof of formula (1) is left as an exercise.


*Secondly, we need that the flavor-spin group $$\begin{align}H~:=~&SU(3)_F\times SU(2)_S\cr
~\cong~&\begin{pmatrix} SU(3)_F \cr & SU(2)_S \cr && 1\end{pmatrix}_{6\times 6}~~\cr
 ~\subseteq~& SU(6)~=:~G\end{align}\tag{2}$$ is a subgroup of $SU(6)$, i.e unitary $6\times 6$ matrices with unit determinant. This means that any irrep of $G$ decomposes via branching rules to a direct sum of irreps for $H$.


*Thirdly, a quark $q$ transforms under $H$ in the fundamental representation $${\bf 3} \otimes {\bf 2} ~\cong~{\bf 6},\tag{3}$$
which is isomorphic to the fundamental irrep of $SU(6)$. We next postulate that the $SU(6)$ group is an approximate symmetry of QCD. The 3 quarks must sit in the fully symmetric tensor product ${\bf 6}^{\odot 3}$. In that way, when the antisymmetric $SU(3)_C$ color singlet ${\bf 1}_A$ is taken into account, the total wave function for the 3 fermions becomes totally antisymmetric, as it should, cf. the Pauli exclusion principle.


*Fourthly, we decompose the tensor product of $H$-irreps into a direct sum of $H$-irreps
$$\begin{align}{\bf 56}_S~\stackrel{(1)}{=}~&{\bf 6}^{\odot 3}
\stackrel{(3)}{\cong}~\left({\bf 3} \otimes {\bf 2} \right)^{\odot 3}\cr
~\cong~&{\bf 3}^{\odot 3}\otimes {\bf 2}^{\odot 3}\oplus {\rm Adj}(SU(3)_F)\otimes {\bf 2} \cr
\stackrel{(1)}{=}~&\underbrace{{\bf 10}_S}_{\begin{array}{c}\text{baryon}\cr\text{decuplet}\end{array}}\otimes \underbrace{{\bf 4}_S}_{j=\frac{3}{2}} \oplus \underbrace{\color{red}{{\bf 8}_M}}_{\begin{array}{c}\text{baryon}\cr\text{octet}\end{array}}\color{red}{\otimes} \underbrace{\color{red}{\bf 2}}_{j=\frac{1}{2}},\end{align}\tag{4}$$
where we have marked the sought-for spin-half octet $\color{red}{{\bf 8}_M \otimes {\bf 2}}$ in red.


*Now, OP wants to identify the baryon octet in terms of the un-symmetrized tensor product
$$\begin{align} {\bf 6}^{\otimes 3} \quad\cong\quad& 
\begin{array}{rcl} [~~]& [~~] & [~~] \end{array}
\quad\oplus~~
2\cdot~~\begin{array}{rl} [~~]&[~~]\cr [~~]\end{array} 
\quad\oplus\quad
\quad\begin{array}{c} [~~]\cr [~~]\cr [~~] \end{array} \cr
\quad\cong\quad&{\bf 56}_S  \oplus 2\cdot {\bf 70}_M  \oplus {\bf 20}_A .\end{align}\tag{5}$$
Similarly, there is OP's original fusion rule for flavor
$${\bf 3}^{\otimes 3}~\cong~{\bf 10}_S  \oplus 2\cdot \color{red}{{\bf 8}_M}  \oplus {\bf 1}_A,\tag{6}$$
and there is a Clebsch-Gordan rule for spin
$${\bf 2}^{\otimes 3}~\cong~{\bf 4}_S  \oplus 2\cdot \color{red}{\bf 2}.\tag{7}$$
Notice the multiplicity of 2 in both eqs. (6) & (7). In terms of the flavor-spin group $H$, there are not just 2 but actually $2\times 2=4$ spin-half octets $\color{red}{{\bf 8}_M \otimes {\bf 2}}$, so why aren't there 3 more baryon octets, cf. OP's title question?
We calculate:
$$ \begin{align} 
({\bf 3}  \otimes & {\bf 2} )^{\otimes 3}\cr
~\cong~&{\bf 3}^{\otimes 3} \otimes {\bf 2}^{\otimes 3}\cr
~\stackrel{(6)+(7)}{\cong}&~\left({\bf 10}_S  \oplus 2\cdot \color{red}{{\bf 8}_M}  \oplus {\bf 1}_A\right) 
\otimes \left({\bf 4}_S  \oplus 2\cdot \color{red}{\bf 2} \right)\cr
 ~\cong~& 
\underbrace{\left({\bf 10}_S \otimes {\bf 4}_S \oplus \color{red}{{\bf 8}_M \otimes {\bf 2}} \right)}_{={\bf 56}_S} \cr
&\oplus 2\cdot \underbrace{\left( {\bf 10}_S \otimes {\bf 2} \oplus {\bf 8}_M \otimes {\bf 4}_S \oplus \color{red}{{\bf 8}_M \otimes {\bf 2}} \oplus {\bf 1}_A \otimes {\bf 2}\right)}_{={\bf 70}_M} \cr
&\oplus \underbrace{\left(  \color{red}{{\bf 8}_M \otimes {\bf 2}} \oplus {\bf 1}_A \otimes {\bf 4}_S\right)}_{={\bf 20}_A}.\end{align} \tag{8}$$
We see that only 1 out of the 4 spin-half octets $\color{red}{{\bf 8}_M \otimes {\bf 2}}$ sits in the fully symmetric flavor-spin tensor product ${\bf 6}^{\odot 3}$ as required by color-confinement. This answers OP's title question.
References:

*

*G. 't Hooft, Introduction to Lie Groups in Physics, lecture notes, p. 58. The pdf file is available here.


*J. Chyla, Quarks, partons and QCD, lecture notes, Section 2.7. The pdf file is available here. (Hat tip: Evans.)
A: I think I have worked out my confusion so I thought I should post it as an answer. The original question was not well-posed; hopefully this will help anyone else who has similar misunderstandings.
Keeping spin in the picture, the space of states for an individual quark is the tensor product of the three-dimensional flavour space with the two-dimensional spin representation of SU(2). However, this is not considered as a representation of SU(3) x SU(2), rather as a representation of SU(6) containing this product as a subgroup. In other words you can rotate flavours into spins and vice versa. As a representation of SU(6), the tensor cube of this standard 6-dimensional representation decomposes into pieces, one of which is the symmetric cube. This is an irreducible 56-dimensional representation. This decomposes under the subgroup SU(3) x SU(2) into a direct sum of two pieces: one is the SU(3)-decuplet tensored with the (4-dimensional) spin-3/2 representation of SU(2), one is the adjoint representation of SU(3) (the octet) tensored with the spin-1/2 representation of SU(2) (and indeed 10 x 4 + 8 x 2 = 56).
When you forget about spin, the spin-1/2 octet that appears here is actually a mixture of terms from the two SU(3)-octets in the decomposition of the tensor cube from the original question. In other words, the actual wavefunction of a proton is a sum of two terms, one involving terms from one SU(3)-octet and one involving terms from the other (both tensored with suitable spin wavefunctions).
I found these notes of Jiří Chýla very helpful in sorting out my misunderstanding.
A: I thought that, perhaps, the solution of the problem would lie in the fact that the two octet representations in which the tensor product 3x3x3 of SU(3) splits are two equivalent representations. That's right what Chyla says in his notes. So one representation can be obtained from the other performing a rotation in flavour space, and the two octets are physically indistinguishable
