What is angular momentum in quantum mechanics? In Quantum Mechanics, we call any triplet of operators $J_i$ (of any representation) that satisfy the commutation conditions $$[J_i, J_j]=i\hbar\epsilon_{ijk}J_k$$ an angular momentum operator.
Why is that? Does this have to do with how they interact with the rotation operators?
The question is not about etymology. To clarify what I am after I will the same question in a different way.
We define a "total angular momentum" as $\hat{L} \otimes 1 + 1 \otimes \hat{S}$. We could also define an operator $\hat{x} \otimes 1 + 1 \otimes \hat{S}$, this would be completely valid from a purely algebraic point of view, but we don't do that. My suspicion is that this has to do with how the operators act under rotations (something is invariant/conserved).
 A: 
In Quantum Mechanics, we call any triplet of operators $J_i$ (of any dimension) that satisfy the commutation conditions $[J_i, J_j]=\epsilon_{ijk}J_k$ an angular momentum operator.


Why is that?

We call them that because the angular momentum operator $\vec r \times \vec p$ satisfies these commutation relations. The rest is a reasonable and experimentally-validated generalization. (E.g., the existence of spin angular momentum explains many experimental facts.)

Does this have to do with how they interact with the rotation operators?

The angular momentum operators certainly are related to the rotation operators, but this is not why we call them that... which is what you asked about. (Spoiler alert: The angular momentum operator is the generator of rotations!)
We use the term "angular momentum" because this is the term we use for $\vec r \times \vec p$ in mechanics.

Update regarding OP's question update:

We define a "total angular momentum" as $\hat{L} \otimes 1 + 1 \otimes \hat{S}$. We could also define an operator $\hat{x} \otimes 1 + 1 \otimes \hat{S}$, this would be completely valid from a purely algebraic point of view, but we don't do that. My suspicion is that this has to do with how the operators act under rotations (something is invariant/conserved).

You will find that the operator
$$
\hat{x} \otimes 1 + 1 \otimes \hat{S}\;,
$$
is not an angular momentum vector, unless $\hat{x}$ is already an angular momentum vector. Therefore, even though you can define such a thing (assuming you have some constants that make the units work out), it is potentially not useful.
A: The answer to your question is something called representation theory. I'll briefly sketch the idea, and encourage you to take a look at Weinberg's The Quantum Theory of Fields Chapter 2, which includes some of the basics here before going into the relativistic setting. As you correctly point out, angular momentum is closely related to rotation symmetry. Now, rotations in three dimensions are elements of the group ${\rm SO}(3)$. If our system has rotational symmetry, we would like the following to happen:

*

*For every rotation matrix $g\in{\rm SO}(3)$ there is one unitary operator $U(g)$ acting on the Hilbert space. The action $U(g)|\psi\rangle$ is the action of a rotation on the state $|\psi\rangle$. Moreover, we demand unitarity of $U(g)$ because then probability amplitudes are preserved, i.e., $\langle \psi|U^\dagger(g)U(g)|\phi\rangle=\langle \psi|\phi\rangle$. This expresses the rotational symmetry.


*If $g,g'\in {\rm SO}(3)$ are two rotations, acting with $U(gg')$ should be the same as acting with $U(g)U(g')$. At this point, one must be careful. Recall that states are defined up to phases, i.e., they are really unit rays in the Hilbert space. This important thing makes the mathematical statement of "$U(gg')$ being the same as $U(g)U(g')$" being $$U(gg')=e^{i\phi(g,g')}U(g)U(g')\tag{1}.$$
This means that $U(g)$ is something we call a projective representation of ${\rm SO}(3)$. If $\phi(g,g')=0$ for all $g,g'\in{\rm SO}(3)$ we would say that $U(g)$ is a unitary representation. Now suppose you wish to classify all the quantum mechanical systems with rotational symmetry. The above remarks means that we should look for a classification of all projective representations of ${\rm SO}(3)$, which is a mathematical problem. This problem, in turn, can be translated into another problem, which is the classification of all unitary representations of the universal cover of ${\rm SO}(3)$, which is the group ${\rm SU}(2)$. This means we can set the phases in (1) to zero by going to this universal cover.
Now, the mathematical problem of finding the unitary representations of ${\rm SU}(2)$ can be solved by looking at the Lie algebra of ${\rm SU}(2)$ which is the same as that of ${\rm SO}(3)$ because one is the universal cover of the other. The reason that we should look to the Lie algebra is the following. Imagine $g = 1+i\epsilon X$ is an element of ${\rm SU}(2)$ that is close to the identity. Then write a corresponding expansion for $U(g)$ $$U(g)=1+i\epsilon T(X)\tag{2},$$
where $T(X)$ is some operator. It is not hard to show that $T(X)$ must be hermitian if $U(g)$ is to be unitary and, moreover, that $$[T(X),T(Y)]=T([X,Y])\tag{3}.$$
This turns $T(X)$ into what we call a Hermitian representation of the Lie algebra $\mathfrak{su}(2)$. Now, the Lie algebra $\mathfrak{su}(2)$ is a three-dimensional vector space, and so we can choose a basis, let's say $\{X_i\}$. So, to specify $T(X)$ we just need to specify operators $J_i = T(X_i)$. The operators $J_i$ are then called the generators of the representation. In terms of these, we can define $U(g)$ for $g\in{\frak su}(2)$ close to the identity. More general elements are then obtained by exponentiation.
So the whole problem we started with: classify all possible rotationally symmetric quantum systems becomes a problem of finding all possible realization of the generators $J_i$, which are constrained by (3). In fact, there is a basis of ${\frak su}(2)$ in which the generating matrices $X_i$ obey $$[X_i,X_j]=i\epsilon_{ijk}X_k\tag{4}.$$
As a result, from (3) the generators of any representation $J_i$ are forced to obey $$[J_i,J_j]=i\epsilon_{ijk}J_k\tag{5}.$$
But this is exactly the definition of angular momenta. This may not be surprising at all. When we study Hamiltonian mechanics we learn that in the classical theory the most accurate characterization of angular momentum is that it is the generator of rotations. So because of all this discussion, angular momentum is defined in the way it is defined, and it is strongly tied to rotational symmetry.
Then, the whole discussion in QM textbooks that lead to a classification of the possible $J_i$ and the possible Hilbert spaces in which they act is, in reality, a derivation of all the irreducible unitary representations of ${\rm SU}(2)$, which are the building blocks for all the other representations. One is really studying, what are the possible systems admitting rotational symmetry.
A: 
In Quantum Mechanics, we call any triplet of operators $J_i$ (of any dimension) that satisfy the commutation conditions $[J_i, J_j]=\epsilon_{ijk}J_k$ an angular momentum operator.

I'm not really sure I agree with that premise. We can define so-called "internal" degrees of freedom that satisfy the commutation relations $[A_i, A_j] = \epsilon_{ijk} A_k$ but are otherwise unrelated to angular momentum. An actual example of this is the isospin. Notice that the linked article says "its quantum mechanical description is mathematically similar to that of angular momentum" and not something like "the isospin operator is an angular momentum operator".
The reason why the $\hat J$ operator is called the total angular momentum operator is that it actually does correspond to the total angular momentum from classical mechanics, in that it is the generator of (overall) rotations and it is conserved (by Noether's theorem) when rotational symmetry is present. For any realistic macroscopic system that can be studied in the laboratory, the distribution of outcomes from measuring $\hat J$ is narrowly peaked and its expected value is the classical angular momentum. $\hat L$ and $\hat S$ are "components" of $\hat J$, insofar as $\hat J = \hat L \otimes 1 + 1 \otimes \hat S$ as noted by the OP, and each is the total angular momentum in the subset of systems where the other vanishes identically, so they, too, bear a name that contains "angular momentum".
Defining "angular momentum operator" in terms of the commutation relations seems to be primarily a pedagogical device to make the student focus on the math instead of trying to physically interpret the operators.
A: For the special case of non-relativistic electron spin, the reasoning for this is that the interaction term of the Hamiltonian is $-\frac{g}{2}\vec{B}\cdot \vec{S}$, where $\vec{S}$ are the Pauli matrices. So you can see that it makes sense to call $\vec{S}$ an "angular momentum" because it couples with magnetic field similar to how orbital angular momentum couples with megnetic field. Also, $\vec{L}+\vec{S}$ is a conserved quantity of the theory.
But in general, the above logic doesn't work. Because photons carry spin too, and they don't couple with magnetic field. The more general reasoning for calling these observables the "angular momenta" is that they're conserved quantities of the theory, pretty much by construction.
In general, we look for representations of the Poincaire group. These commutation relations are part of the definition of the Poincare algebra. That these operators commute with the generator of time translations, is also part of the definition of the Poincaire algebra. So you get various Quantum Field Theories with various operators satisfying these commutation relations, which are also conserved quantities of the theory by construction.
A: 
In Quantum Mechanics, we call any triplet of operators $J_i$ (of any dimension) that satisfy the commutation conditions $[J_i, J_j]=\epsilon_{ijk}J_k$ an angular momentum operator.

Because for the following two reasons:

*

*$\hat{J} = \hat{L} \otimes 1 + 1 \otimes \hat{S}$.

$L_i$ clearly denotes the (orbital) angular momentum as $L_i = \epsilon_{ijk}r_jp_k = -i\hbar\epsilon_{ijk}x_j\partial_k$ (in position basis) and if you calculate, it will then satisfy the commutation relation $[L_i,L_j] = \epsilon_{ijk}L_k$ ... (I)
Now come to the point of (spin) angular momentum $S_i$. Long after the Stern-Gerlach Experiment, theory of the spin angular momentum was being developed (Outcome of the Stern-Gerlach Experiment says that the deflection of the silver beams must have an intrinsic angular momentum, thus $S_i$ corresponds to an angular momentum in principle; this was historically the first argument that the spin corresponds to an angular momentum). But as there is no classical analogue of 'quantum spin', so no such formulation (like $\hat{L} = \hat{r} \times \hat{p}$) was available from the classical mechanics knowledge. Now as you know, most of the formulation of QM was being done comparing that's analogous classical formulation, so in this case of spin angular momentum, what is the way out. The simplest way is to derive an 'algebra' similar to the orbital angular momentum (as at the end of the day, spin results an angular momentum also). So after this, the building block of the 'spin' angular momentum algebra became, $[S_i,S_j] = \epsilon_{ijk}S_k$ ... (II)
[We accept this algebra of $S_i$ because it is well fitted with experimental observations and no counter-explanation of this theory has yet been discovered.]
Add (I) and (II),
$[J_i,J_j] = \epsilon_{ijk}J_k$.


*Here is the answer of:


Does this have to do with how they interact with the rotation operators?

Infinitesimal rotation is the generator of $L_i$ (and hence $J_i$).
Outline of the proof of this statement:
Consider rotation around z-axis associated with the rotation matrix becomes, $R_{z}(\theta)$.
$\therefore \vec{r'} = \hat R_{z}(\theta) \vec{r}$
where, $\hat R_{z}(\theta) = 
  \begin{pmatrix}
    cos\theta & -sin\theta & 0\\
    sin\theta & cos\theta & 0\\
    0 & 0 & 1
  \end{pmatrix}$; $\hat R_{z}(\theta_1 + \theta_2) = \hat R_{z}(\theta_1) + \hat R_{z}(\theta_2)$.
Now consider $\psi(\vec r) \longrightarrow \psi(\vec{r'}) = \hat R_{z}(\theta) \psi(\vec{r})$ and an infinitesimal rotation $\delta\theta$,
$\therefore \hat R_{z}(\delta\theta) = 
  \begin{pmatrix}
    1 & -\delta\theta & 0\\
    \delta\theta & 1 & 0\\
    0 & 0 & 1
  \end{pmatrix}$
So after few lines, $\delta x = -y\delta\theta$, $\delta y = x\delta\theta$, $\delta z = 0$.
Now, $\psi(x + \delta x, y + \delta y, z + \delta z) = \psi(x, y, z) + \dfrac{\partial\psi}{\partial x}\delta x + \dfrac{\partial\psi}{\partial y}\delta y = \hat R_{z}(\delta\theta)\psi(x,y,z)$.
Finally, $\hat R_{z}(\delta\theta) \equiv \hat{I} + \dfrac{i}{\hbar}\delta\theta.\hat L_z$ [QED]
Note: For more clear concepts and primary intuitions on this topic, you may go through the relevant chapters of the Quantum Mechanics book by David J. Griffiths.
