Tensor Operators Motivation.
I was recently reviewing the section 3.10 in Sakurai's quantum mechanics in which he discusses tensor operators, and I was left desiring a more mathematically general/precise discussion.  I then skimmed the Wikipedia page on tensor operators, and felt similarly dissatisfied.  Here's why
In these discussions, one essentially defines an indexed set of operators $T_{i_1\cdots i_k}$ to be a "cartesian" tensor operator of rank $k$ provided
$$
  U(R)\, T_{i_1\cdots i_k}\, U^\dagger(R) = R_{i_1}^{\phantom{i_1}j_1}\cdots R_{i_1}^{\phantom{i_1}j_1}T_{j_1\cdots j_k}
$$
for each rotation $R\in\mathrm{SO}(3)$ where $U$ is some unitary representation of $\mathrm{SO}(3)$ acting on a Hilbert space (usually that of some physical system whose behavior under rotations we with to study).  Similarly one defines a "spherical" tensor operator of rank $n$ as an indexed set of operators $T^{(n)}_{q}$ with $-n<q,q'<n$ for which
$$
  U(R)\,T_q^{(n)}\,U^\dagger(R) = \sum_{q'=-n}^n D_{q'q}^{(n)}(R)T_{q'}^{(n)}
$$
where $D^{(n)}$ is the irreducible representation of $\mathrm{SO}(3)$ of dimension $n$.
Based on these standard definitions, I would think that one could define something less "coordinate-dependent" and extended to representations of any group, not just $\mathrm{SO}(3)$, as follows.

Candidate Definition. Let a group $G$ be given.  Let $U$ be a unitary representation of $G$ on a Hilbert space $\mathcal H$, and let $\rho$ be a representation of $G$ on a finite-dimensional, real or complex vector space $V$.  A $k$-multilinear, linear operator-valued function $T:V^k\to \mathrm{Lin}(\mathcal H)$ is called a tensor operator relative to the pair of representations $U$ and $\rho$ provided
  \begin{align}
  U(g) T(v_1, \dots, v_k) U(g)^\dagger = T(\rho(g)v_1, \dots, \rho(g)v_k)
\end{align}
  for all $g\in G$ and for all $v_1, \dots, v_k\in V$.

Notice that if a basis $u_1, \dots, u_N$ for $V$ is given, and if we define the components $T_{i_1,\dots i_k}$ of $T$ in this basis by
\begin{align}
  T_{i_1 \dots i_k} = T(u_{i_1}, \dots, u_{i_k})
\end{align}
and if $\rho(g)_i^{\phantom ij}$ denotes the matrix representation of $\rho(g)$ in this basis, then by using multilinearity the defining property of a tensor operator can be written as follows
\begin{align}
  U(g) T_{i_1\cdots i_k} U^\dagger(g) = \rho(g)_{i_1}^{\phantom {i_1}j_1}\cdots \rho(g)_{i_k}^{\phantom {i_k}j_k} T_{j_1\cdots j_k}
\end{align}
So this definition immediately reproduces the cartesian tensor definition above if we take, $V =\mathbb R^3$, $G=\mathrm{SO}(3)$, and $\rho(R) = R$, and similarly for the spherical tensor definition if we take $V=\mathbb C^{2n+1}$, $G=\mathrm{SO}(3)$, $\rho = D^{(n)}$ and $k=1$.
Question. 
Is the sort of object I just defined the "proper" formalization/generalization of the notion of tensor operators used in physics; it seems to contain the notion of tensor operator used in the physics literature?  Is there any literature on the sort of object I define here?  I would think that the answer would be yes since this sort of thing seems to me like a natural generalization a mathematically-minded physicist might like to study.
 A: The definition suggested by joshphysics and clarified by Qmechanic already exists in the literature under then name of representation operator. This is discussed in, e.g., Sternberg's  Group Theory and Physics, as well as the somewhat more elementary text An Introduction to Tensors and Group Theory for Physicists by Jeevanjee.
A: OP's candidate definition is a direct transcription of the tensor operator notion used in physics (and e.g. in Sakurai section 3.10) into a manifestly coordinate-independent mathematical construction. Tensor operators are e.g. used in the Wigner-Eckart theorem.
In this answer we suggest the following slight generalization of OP's candidate definition. Let the following five items be given:

*

*Let $G$ be a group.


*Let $H$ be a complex Hilbert space.


*Let $\rho: G \to GL(V,\mathbb{F})$ be a group representation.


*Let $R:G \to B(H)$ be a group representation.


*Let $T:V\to L(H;H)$ be a linear map.

Definition. Let us call $T$ for a $G$-equivariant map if
$$\begin{align} \forall g\in G, v\in V: &\cr  
T(\rho(g)v)~=~& {\rm Ad}(R(g))T(v)\cr~:=~&R(g)\circ T(v)\circ R(g)^{-1}. \end{align}\tag{*} $$

OP's candidate definition may be viewed as a special case of definition (*). For instance, if $\rho_0: G \to GL(V_0,\mathbb{F})$ is a group representation, then one may let $\rho: G \to GL(V,\mathbb{F})$ in point 3 be the tensor product representation $\rho=\rho_0^{\otimes m}$ with vector space
$$V~=~V_0^{\otimes m}~=~\underbrace{V_0\otimes \ldots \otimes V_0}_{m \text{ factors}}.$$
A: In first chapter of Lie Groups for Pedestrians by Lipkin, a method of generalization of irreducible tensor operators (and other features of the quantum mechanical angular momentum algebra) is given. 
The statement is that as long as one can find a finite number of operators $X_\rho$ satisfying analogous commutation relations to those of the angular momentum operators in quantum mechanics, i.e. 
$$[X_\rho,\;X_\sigma]=C^\tau_{\rho\sigma}X_\tau,$$
it is always possible to find irreducible tensor operators. One can then, in analogy to $J_z$, choose one (or several) operators to be diagonal in the desired representation. Furthermore, one can extract the analogy of the ladder operators $J_x\pm iJ_y$. 
For angular momentum ($SO(3)$), irreducible tensor operators are given in terms of the relation
$$[J_z,T_{kq}]=qT_{kq},$$ 
where $q$ is the number of components and $k$ is the rank of the tensor. There are $2k+1$ values for $q$, which ranges from $-k$ to $k$. 
Analogous tensor operators can be constructed starting from any algebra of the above form. Note that the crucial object is the Lie algebra, not the Lie group, which can be formulated as the group of continuous transformations given by
$$\psi^\prime=(1+i\epsilon X_\rho)\psi.$$
This is not a rigorous answer since I haven't worked out the proof myself. I can only recommend you to read the book.  
A: If you want a more geometric viewpoint, this link is a good start. It's the first chapter of Applications of Classical Physics by Blandford and Thorne. The advantage of the formulation is that the transformation laws for tensors can be derived naturally from the transformation laws for vectors. Then, if you want your tensors to transform a certain way, just modify your transformation laws of your vectors.
Here's a very quick summary (for Cartesian tensors, ie, our vectors live in euclidean space): a rank-$k$ tensor is defined as a function from $k$ vectors to a real number, for example, a rank-two tensor might be written as
$$
T = T(\_,\_)
$$
note that a vector can be regarded as a rank-1 tensor, $v(w) = v \cdot w$, and hence a rank-two tensor can also be regarded as a function from vectors to vectors, which is probably relevant for the specific application of tensors in Sakurai. Tensor product is defined as the product of the functions
$$
S(\_,\_) \otimes T(\_,\_,\_) = S(\_,\_)T(\_,\_,\_)
$$
Once you choose a basis, you can write the tensor components
$$
T = T_{ijk} e_i e_j e_k
$$
where $i,j,k$ are implicitly summed over. From this one can derive how the usual formula for how components $T_{ijk}$ transform under rotation. As an example, suppose $T$ is rank 2 and treat it as a function from vectors to vectors; then we can view the $T_ijk$ as components of 3 by 3 matrix. If $T$ carries a vector $v$ to $Tv$, then $T'$ must carry $Rv$ to $R(Tv)$, so $T' = RTR^{-1}$.
A: The generalization of the "spherical tensor harmonics" of quantum mechanics to a general case of a compact Lie group is given as follows:
Let $G$ be a compact Lie group and $H$ be a closed subgroup, then, the
Hilbert space of square integrable functions on $G/H$ (which may be taken as the eigenfunctions of the Killing Laplacian) is a direct sum of $G$-representations called "spherical representations". These representations are characterized by having an  $H$ singlet. Please see, for example, appendix B of HARMONIC ANALYSIS AND PROPAGATORS ON HOMOGENEOUS SPACES by Camporesi. This definition generalizes (and was named after) the spherical harmonics of quantum mechanics. In this case $G=SU(2)$, $H=U(1)$ and $G/H = SU(2)/U(1)$. The sphericality condition implies that the spherical harmonics can only be representations containing a $U(1)$ singlet, thus must be of integer spin.
A: A tensor operator is a collection of operators that transform irreducibly under conjugation by group elements, i.e. that satisfy precisely the second condition of the OP. The individual elements of the set are the components of the tensors. 
Irreducibility is not essential but an additional obvious requirement when dealing with groups for which representations are always fully reducible. 
This can be generalized to any group - the group doesn't even need to be continuous.  All you need is a group action on the set of components of the tensor operator.
A: I don't think that the candidate definition is sound because the notation 
$T(v_{1},\ldots ,v_{k})$ implies - by analogy with mathematicians' notation for tensors - that $T(v_{1},\ldots ,v_{k})$ is an operator which transforms trivially; but it clearly does not transform trivially as the RHS of the candidate definition shows. Nevertheless, there is a tensor which transforms trivially under the group G. For simplicity, consider the case $k=1$ and furthermore suppose that the tensor operators $T_{i}$ are Hermitian. So far, the indices of the Hermitian operators $T_{i}$ have been suppressed in the question and answer; putting in the indices, we are dealing with ordinary tensors,
$$
T_{i\ \ B}^{\ \bar{A}} \ .
$$
Suppose that these tensors transform trivially under the group G. In other words,
$$
[D(g^{-T})]_{i}^{\ \ k}[D(g^{-\dagger})]^{\bar{A}}_{\ \ \bar{C}}[D(g^{-T})]_{B}^{\ \ D}
T_{k\ \ D}^{\ \bar{C}}=T_{i\ \ B}^{\ \bar{A}}
$$
Multiply both sides by $[D(g)]^{i}_{\ \ l}$.
$$
[D(g^{-\dagger})]^{\bar{A}}_{\ \ \bar{C}}[D(g^{-T})]_{B}^{\ \ D}
T_{l\ \ D}^{\ \bar{C}}=T_{i\ \ B}^{\ \bar{A}}[D(g)]^{i}_{\ \ l}
$$
Suppressing the indices of the Hermitian operators recovers the standard definition of a set of tensor operators $T_{i}$,
$$
D(g^{-\dagger})T_{l}D(g^{-1})=T_{i}[D(g)]^{i}_{\ \ l}
$$
In other words, the ordinary tensor,
$$
T_{i\ \ B}^{\ \bar{A}} \ .
$$
transforms trivially under the group and a less coordinate-dependent definition needs to proceed from the fact that this tensor transforms trivially.
A further minor caveat is that the group matrices $[D(g)]^{A}_{\ \ B}$ are not necessarily unitary; if the indices $A,B$ are Lorentz spinor indices, the group matrices are the defining rep of SL(2,C); these are only unitary if the Lorentz transformation is a spatial rotation.
