Matrix element and Dirac notation If
$$ 
T=
\left[
\begin{array}{cccc}
   e^{\beta J} & e^{-\beta J} \\
   e^{-\beta J} & e^{\beta J} \\
\end{array} \right]
$$
and
$$Z = \sum_{S_i=\pm 1} ... \sum_{S_N=\pm 1} \exp{\beta J(\vec{S_1}\vec{S_2}+\vec{S_2}\vec{S_3}+...+\vec{S_{N-1}}\vec{S_N}+\vec{S_N}\vec{S_1})}
$$
Then why can we say that
$$Z = \sum_{S_i=\pm 1} ... \sum_{S_N=\pm 1} \langle S_1|T|S_2\rangle\langle S_2|T|S_3\rangle...\langle S_N|T|S_1\rangle ?
$$
 A: $\newcommand{\e}{\boldsymbol=}$
$\newcommand{\p}{\boldsymbol+}$
$\newcommand{\m}{\boldsymbol-}$
$\newcommand{\gr}{\boldsymbol>}$
$\newcommand{\les}{\boldsymbol<}$
$\newcommand{\greq}{\boldsymbol\ge}$
$\newcommand{\leseq}{\boldsymbol\le}$
$\newcommand{\plr}[1]{\left(#1\right)}$
$\newcommand{\blr}[1]{\left[#1\right]}$
$\newcommand{\lara}[1]{\langle#1\rangle}$
$\newcommand{\lav}[1]{\langle#1|}$
$\newcommand{\vra}[1]{|#1\rangle}$
$\newcommand{\lavra}[2]{\langle#1|#2\rangle}$
$\newcommand{\lavvra}[3]{\langle#1|\,#2\,|#3\rangle}$
$\newcommand{\x}{\boldsymbol\times}$
$\newcommand{\qqlraqq}{\qquad\boldsymbol{-\!\!\!-\!\!\!-\!\!\!\longrightarrow}\qquad}$
Consider two complex $n\m$vectors expressed also as kets
\begin{equation}
\mathbf x\e 
\begin{bmatrix}
x_1 \vphantom{\dfrac{a}{b}}\\
x_2 \vphantom{\dfrac{a}{b}}\\
\vdots \vphantom{\dfrac{a}{b}}\\
x_n \vphantom{\dfrac{a}{b}}\\
\end{bmatrix}\e \vra{\mathbf x}\qquad \texttt{and} \qquad
\mathbf y\e 
\begin{bmatrix}
y_1 \vphantom{\dfrac{a}{b}}\\
y_2 \vphantom{\dfrac{a}{b}}\\
\vdots \vphantom{\dfrac{a}{b}}\\
y_n \vphantom{\dfrac{a}{b}}\\
\end{bmatrix}\e \vra{\mathbf y}\quad \in \mathbb C^n
\tag{01}\label{01}  
\end{equation}
Complex conjugating and transposing these one-column matrices we obtain the bras
\begin{equation}
\mathbf x^{\boldsymbol*}\e 
\begin{bmatrix}
\overline x_1 & \overline x_2 & \cdots & \overline x_n \vphantom{\dfrac{a}{b}}\\
\end{bmatrix}\e \lav{\mathbf x}\quad \texttt{and} \quad
\mathbf y^{\boldsymbol*}\e 
\begin{bmatrix}
\overline y_1 & \overline y_2 & \cdots & \overline y_n \vphantom{\dfrac{a}{b}}\\
\end{bmatrix}\e \lav{\mathbf y}
\tag{02}\label{02}  
\end{equation}
Their usual inner product in $\,\mathbb C^n\,$ is
\begin{equation}
\overline x_1\,y_1\p\overline x_2\,y_2\p\cdots\overline x_n\,y_n\e
\begin{bmatrix}
\overline x_1 & \overline x_2 & \cdots & \overline x_n \vphantom{\dfrac{a}{b}}\\
\end{bmatrix}
\begin{bmatrix}
y_1 \vphantom{\dfrac{a}{b}}\\
y_2 \vphantom{\dfrac{a}{b}}\\
\vdots \vphantom{\dfrac{a}{b}}\\
y_n \vphantom{\dfrac{a}{b}}\\
\end{bmatrix}\e \lavra{\mathbf x}{\mathbf y}
\tag{03}\label{03}  
\end{equation}
Given a $\,n\times n\,$ complex matrix $\,\mathrm A\,$
\begin{equation}
\mathrm A\e 
\begin{bmatrix}
a_{11} & a_{11} & \cdots & a_{1n} \vphantom{\dfrac{a}{b}}\\
a_{21} & a_{22} & \cdots & a_{2n}  \vphantom{\dfrac{a}{b}}\\
\vdots & \vdots & \vdots & \vdots\vphantom{\dfrac{a}{b}}\\
a_{n1} & a_{n2} & \cdots & a_{nn}\vphantom{\dfrac{a}{b}}\\
\end{bmatrix}
\tag{04}\label{04}  
\end{equation}
the notation $\,\lavvra{\mathbf x}{\mathrm A}{\mathbf y}\,$ is the inner product of the vectors $\,\mathbf x\,$ and $\,\mathrm A\mathbf y\,$ expressed by matrices as
\begin{equation}
\lavvra{\mathbf x}{\mathrm A}{\mathbf y}\e 
\begin{bmatrix}
\overline x_1 & \overline x_2 & \cdots & \overline x_n \vphantom{\dfrac{a}{b}}\\
\end{bmatrix}
\begin{bmatrix}
a_{11} & a_{11} & \cdots & a_{1n} \vphantom{\dfrac{a}{b}}\\
a_{21} & a_{22} & \cdots & a_{2n}  \vphantom{\dfrac{a}{b}}\\
\vdots & \vdots & \vdots & \vdots\vphantom{\dfrac{a}{b}}\\
a_{n1} & a_{n2} & \cdots & a_{nn}\vphantom{\dfrac{a}{b}}\\
\end{bmatrix}
\begin{bmatrix}
y_1 \vphantom{\dfrac{a}{b}}\\
y_2 \vphantom{\dfrac{a}{b}}\\
\vdots \vphantom{\dfrac{a}{b}}\\
y_n \vphantom{\dfrac{a}{b}}\\
\end{bmatrix}
\tag{05}\label{05}  
\end{equation}
Under this spirit you could look at the $\,S_k\,$ as $2\times 1$ matrices and more precisely
\begin{equation}
S_k\e
\left.
\begin{cases}
\begin{bmatrix}
1 \vphantom{\dfrac{a}{b}}\\
0 \vphantom{\dfrac{a}{b}}\\
\end{bmatrix} \texttt{for} \p 1\\
\\
\begin{bmatrix}
0 \vphantom{\dfrac{a}{b}}\\
1 \vphantom{\dfrac{a}{b}}\\
\end{bmatrix} \texttt{for} \m 1
\end{cases}\right\}
\tag{06}\label{06}  
\end{equation}
(note : this reminds us the up and down states of a spin-1/2 particle or the up and down quarks of isospin-1/2 particle).
So  if for example $\,S_3\e\m 1\,$ and $\,S_8\e\p 1\,$ then
\begin{equation}
\lavvra{S_3}{\mathrm T}{S_8}\e 
\begin{bmatrix}
0 & 1 \vphantom{\dfrac{a}{b}}\\
\end{bmatrix}
\begin{bmatrix}
t_{11} & t_{11}  \vphantom{\dfrac{a}{b}}\\
t_{21} & t_{22}  \vphantom{\dfrac{a}{b}}
\end{bmatrix}
\begin{bmatrix}
1 \vphantom{\dfrac{a}{b}}\\
0 \vphantom{\dfrac{a}{b}}
\end{bmatrix}\e
\begin{bmatrix}
0 & 1 \vphantom{\dfrac{a}{b}}\\
\end{bmatrix}
\begin{bmatrix}
t_{11} \vphantom{\dfrac{a}{b}}\\
t_{21} \vphantom{\dfrac{a}{b}}
\end{bmatrix}\e t_{21}
\tag{07}\label{07}  
\end{equation}
For the rest look in the other till now two answers.
A: 
Then why can we say that:

Because, each $S_i$ can only take on two values: +1 or -1
For example, if $S_1 = +1$ and $S_2 = +1$ then the $e^{\beta J S1S2}$ is ${e^{\beta J}}$, which is exactly what the ++ matrix element of $T$ says.
As another example, If $S_1 = +1$ and $S_2 = -1$ then the $e^{\beta J S1S2}$ is ${e^{-\beta J}}$, which is exactly what the +- matrix element of $T$ says.
And so on.

Update:
In the Dirac bra/ket notation, $\left < S | T | S' \right >$ is a number that depends on S and S'. (Note: We could use a different symbol to denote $T$ as a stand-alone matrix vs $T$ when it is in the Dirac bra/key notation, but in this case we dont.)
For example, when S=+ and S'=+:
$$
\left < + | T | + \right > = e^{\beta J}
$$
For example, when S=+ and S'=-:
$$
\left < + | T | - \right > = e^{-\beta J}
$$
For example, when S=- and S'=+:
$$
\left < - | T | + \right > = e^{-\beta J}
$$
For example, when S=- and S'=-:
$$
\left < - | T | - \right > = e^{-\beta J}
$$
And we write these four possible values together in a matrix, whose indices span + and - in both directions.
A: Because, with $S_i$ taking values $\pm 1$, we have
$$
\langle S_1|e^{\beta J {S}_i  {S}_{i+1}}| S_2\rangle=
\left[\matrix{e^{\beta J}& e^{-\beta J} \cr e^{-\beta J} &e^{\beta J}}\right]_{S_1,S_2}
$$
where the subscript on the matrix means the appropriate matrix entry.
