Operator-valued vectors and representation theory Let $G$ be a Lie group and $\pi : G\to GL(V)$ a finite-dimensional representation of $G$ in the vector space $V$. For every $g\in G$ we have a linear transformation $\pi(g) : V\to V$. Being linear, if we give a basis $\{E_a\}$ of $V$ we can associate a matrix to $\pi(g)$: $$\pi(g)E_a = \pi_a^{\phantom{a}b}(g) E_b\tag{1}$$
Linearity of $\pi(g)$ then allows us to do the following: $$\pi(g)v = \pi(g)[ v^a E_a]=v^a \pi_a^{\phantom{a}b}(g)E_b\tag{2}$$
So if we agree that the components are all in the same basis, $\pi(g)$ takes a vector $v$ with components $v^a$ to a vector $\pi(g)v$ with components $$[\pi(g)v]^b=v^a\pi_a^{\phantom{a}b}(g)\tag{3}.$$
Now it is then common in Quantum Mechanics that we wish to see a tuple of $\dim V$ operators, $A^a$, as a sort of "operator-valued elements of $V$". In other words, it seems we would like to make sense of something of the form $$A = A^a E_a\tag{4}$$
and allow $\pi(g)$ to act upon $A$ to produce $\pi(g)A$.
What one usually does is demand there be a unitary representation $\Pi : G\to {\rm U}({\cal H})$ with the property that: $$\Pi(g)^{-1}A^b\Pi(g)=A^a \pi_a^{\phantom{a}b}(g)\tag{5}.$$

*

*What is the proper mathematical way to say that a tuple $A^a$ are the components of a single "operator-valued element of $V$" $A$ as in (4)? This seems not obvious to me because (4) is meaningless since operators $A^a$ cannot multipy the basis vectors $E_a$ of $V$.
Moreover, (5) does not seem as such a definition for me. The vector space $V$ is independent of the representation. So we should first be able to define what an "operator-valued element of $V$ is* (make sense of (4)) and later tell how $\pi(g)$ can act on such objects (make sense of (5)).


*Given that we know how to make sense of an "operator-valued element of $V$" why intuitively is (5) the right way to allow $\pi(g)$ to act on such objects? Eq. (5) seems to be saying that the components of $\pi(g)A$ are $$[\pi(g)A]^b= \Pi(g)^{-1}A^b \Pi(g).$$
Why this is the right way to allow a representation $\pi : G\to GL(V)$ on $V$ to act on operator-valued vectors in $V$?
My guess for (1) is that an operator-valued element of $V$ should be a tensor product: $$A = A^a\otimes E_a$$
and that we should take $\pi(g)$ to act upon it by acting on the $V$ factor: $$\pi(g) A = A^a\otimes \pi(g)E_a = (\pi_a^{\phantom{a}b}(g)A^a)\otimes E_b,$$
but then again, I don't see why we should demand there be a unitary representation $\Pi(g)$ so that (5) holds.
 A: Let us not be so abstract as to lose all physical terminology. What you're talking about here is a consistency condition for fields being quantized. 
A classical field $A^\mu$ transforms in some finite-dimensional representation $\pi: V \to V$ of some symmetry group $G$ (e.g. the Lorentz group) as $(\pi(g)\phi)^\mu = \pi(g)^\mu_\nu \phi^\nu$.
In quantum physics, this symmetry group gets represented on a Hilbert space as $U: H \to H$. Since the components of the quantum field are promoted to operators, they transform like all operators do under this transformation: $A^\mu \mapsto U^\dagger(g)A^\mu U(g)$.
Eq. (5) is a condition that "quantization" commutes with "apply $G$", i.e. the result of doing the transformation on the classical field and then interpreting it as an operator is the same as transforming the field operators themselves.
If you want to state rigorously in which space eq. (5) holds, then this is simply the tensor product $V\otimes \mathcal{O}(H)$, where $\mathcal{O}(H)$ is some suitable space of operators on $H$ that contains the quantized $\phi^\mu$.
A: The situation  you describe exactly arises when representing a finite dimensional real Lie group $G$ with a strongly continuous representation (finite or infinite dimensional)
$$G\ni g \mapsto U_g\:.\tag{1}$$
 In this case one focus on the natural representation the group has on its Lie algebra:
$$L_g : \mathfrak{g} \to \mathfrak{g}$$
defined fruom the requirement 
$$g e^{t s} g^{-1} = e^{tL_gs}$$
(taking the derivative for $t=0$ of both sides). 
The representation $G\ni g \mapsto L_g$ is finite dimensional since the Lie algebra is finite dimensional,  thus it is constructed by matrices. 
$$L_g s_k = \sum_r A_k^r(g) s_r$$
for every basis $s_1,\ldots, s_n$ of the Lie algebra $\mathfrak{g}$.
From known theorems on strongly continuous  unitary representations,  the unitary operators in (1) can be written, in a neighborhood of the identity, 
$$U_g = \exp(-i \sum_r a^k S_k)$$
where all linear combinations of the operators $S_k$ generally defined in an infinite dimensional Hilbert space where (1) exists, are essentially self adjoint on a common invariant (also for the representation $U$) dense domain and the map $dU: s_k \mapsto -iS_k$ induces a Lie algebra representation on that space. This domain is known as the Garding domain (another similar domain is that discovered by Nelson).
$S_k$ is here  the generator, in the sense of the  Stone theorem, of the one parameter group $U_{e^{ts_k}}$.
It is now easy to prove that 
$$U_g  S_k U_g^{-1} = \sum_r A_k^r(g) S_r$$
is true on the said domain.
What you call operator valued elements are nothing but the elements in the image of a Lie algebra representation $dU$ of the Lie algebra $\mathfrak{g}$ induced by a strongly continuos unitary representation of the Lie group $G$.
A: The notion you are looking for is that of a representation operator.
Given a representation $(\Pi_0, G)$ of a group G on a Vector space V as well as a unitary representation $(\Pi, G)$ of the same group on a Hilbert space $\mathcal{H}$, a representation operator is a linear map $\rho: V \rightarrow \mathcal{H}$ satisfying 
$$\rho(\Pi_0(g) v) = \Pi(g) \rho(v)\Pi(g)^{-1} \ \forall g \in G, v \in V.$$
First of all such a linear map makes precise the concept of a "dot product with operators". For example given three Operators $A_1, A_2, A_3$ on some Hilbert space H let us define $\rho: \mathbb{R}^3 \rightarrow H$ where $\rho(e_i) = A_i$ and $\{e_i\}$ is a basis of $\mathbb{R}^3$. The product $A = v_i A_i$ is now replaced by the simple object $\rho(v)$. 
I Hope this answers Part I of your Question, where you wrote:

What is the proper mathematical way to say that a tuple $_$ are the components of a single "operator-valued element of  "  as in (4)? This seems not obvious to me because (4) is meaningless since operators $_$ cannot multiply the basis vectors $_a$ of .

Now on to Part II: why is the above definition the right way to let representations act on each other.
The key here is that the transformation properties of the representation operator are equivalent to postulating the existence of a symmetry in QM. To make this clearer imagine a single spin-$\frac{1}{2}$ system. If I measure its spin along the z axis I use $\sigma_z$, if I measure it along the y-axis I use $\sigma_y$ and if I measure it along an arbitrary axis $\hat{n}$ I use $\hat{n}_i \sigma_i = \rho(\hat{n})$. Now the following fact should be obvious:

Measuring an x-Eigenstate in the x-direction gives the same result as measuring a z-Eigenstate in the z-direction.

More mathematically (the above sentence is encoded in the center equals sign):
$$\langle x |\rho(\Pi_o(R_{xz})e_z) | x \rangle =  \langle x |\rho(e_x) | x \rangle = \langle z |\rho(e_z) | z \rangle = \langle x |\Pi(R_{zx})^\dagger \rho(e_z) \Pi(R_{zx})| x \rangle  $$
Where $R_{xz}$ denotes the rotation rotating the z-axis into the x-axis while holding the y-axis fixed, $\Pi_0$ is the fundamental representation of SO(3) and $\Pi$ is a unitary SO(3) representation on $\mathcal{H}$.
Since this statement holds for arbitrary states we obtain
$$\rho(\Pi_o(R_{xz})e_z) =  \Pi(R_{xz}) \rho(e_z) \Pi(R_{xz})^\dagger$$
(notice the change $x \leftrightarrow z $ on the RHS and the change of the location of the $\dagger$) or more generally
$$\rho(\Pi_o(g) v) =  \Pi(R_{g}) \rho(v) \Pi(g)^{\dagger} \ \forall g \in SO(3)$$
which is equal to the above definition by unitarity of $\Pi$.
This should have answered Part II of your question.
Another interesting example of the above construction is the relativistic field operator which satisfies
$$ \phi(\Lambda x + a) = U(\Lambda, a) \phi(x) U(\Lambda, a)^\dagger$$
or for a more complicated field
$$\phi \big((\Lambda^{-1})^{ \mu}_{\nu} e^\nu, \Lambda x + a \big) = U(\Lambda, a) \phi(e^\mu, x) U(\Lambda, a)^\dagger $$
where commonly the notation $\phi(e^\mu, x) = \phi^\mu(x)$ is employed.
These equations would translate to plain english as something like:

Measuring the value of the field operator in a boosted frame gives the same value as measuring a boosted state it in a non-boosted frame. It is therefore impossible to determine which measurement was made and thus Lorentz symmetry holds in our theory.

