Proper conceptualization & notation for vectors, $n$-tuples, and matrices in physical space

Question

This is a fairly basic question that I may be making longer than necessary. But it has plagued me for some time. It is essentially this: In what space do abstract physical vectors like a velocity vector "live"? In what space do the various n-tuple representations of the abstract vector live? and in what space the various $n\times 1$ column vector representations live? By "space" I mean things like vector spaces or manifolds.

Example: Let's keep it simple and limit consideration to just basic flat 3-space and forget about things like covariant and contravariant indices. Consider some vector $\vec{\mathbf{u}}$. let's say it is the instantaneous velocity vector of a bird but it doesn't matter. We can expand this vector in any basis we like as (summation over $i=1,2,3$):

$$\vec{\mathbf{u}}=u_i\hat{\mathbf{e}}_i=u'_i\hat{\mathbf{e}}'_i$$

for arbitrary bases $\hat{\mathbf{e}}_i$ and $\hat{\mathbf{e}}'_i$ (or any other basis we can conceive of in flat 3-space) which we'll take to be orthonormal. The above $u_i=\vec{\mathbf{u}}\cdot\hat{\mathbf{e}}_i$ and $u_i'=\vec{\mathbf{u}}\cdot\hat{\mathbf{e}}'_i$ are simply the coordinates of the vector $\vec{\mathbf{u}}$ in certain bases. But the vector $\vec{\mathbf{u}}$ cares not at all about these coordinates. It exists regardless of any coordiante system or basis we impose on it.

But, given some basis, we can then define a 3-tuple or, similalrly, a column vector, to represent the vector $\vec{\mathbf{u}}$ in this basis,

$$\mathbf{u}=(u_1,u_2,u_3) \; \in\mathbb{R}^3 \qquad\qquad [\mathbf{u}]= \begin{bmatrix} u_1 \\ u_2 \\ u_3 \end{bmatrix} \in\mathbb{R}^3 $$

(I have omitted the arrow over top to distinguish these from $\vec{\mathbf{u}}$) I often see books and papers simply define the vector itself in one of the above ways as a 3-tuple or column vector. But these are not the vector $\vec{\mathbf{u}}$ are they? They are representations (charts) of $\vec{\mathbf{u}}$ in certain bases. There is only one vector but an infinite number of 3-tuples and an infinite number of column vectors one can use to represent it and they are meaningless unless a basis is specified along with them. For instance, the 3-tuple and column vector representation of the same vector but in some other basis (say, the principle axis of some other flying bird) would be

$$\mathbf{u}'=(u'_1,u'_2,u'_3) \; \in\mathbb{R}^3 \qquad\qquad [\mathbf{u}']= \begin{bmatrix} u'_1 \\ u'_2 \\ u'_3 \end{bmatrix} \in\mathbb{R}^3 $$

where $u_i \neq u'_i$. And if $\dot{\vec{\mathbf{u}}}$ is the time-derivative of the vector $\vec{\mathbf{u}}$, then $\dot{\mathbf{u}}=(\dot{u}_1,\dot{u}_2,\dot{u}_2)$ is, in general, not the 3-tuple representation of $\dot{\vec{\mathbf{u}}}$ (unless the basis asociated with he 3-tuple $\mathbf{u}$ happens to be an inertial basis).

my actual question: I have written that the 3-tuples $\mathbf{u}$ and $\mathbf{u}'$ "live" in $\mathbb{R}^3$. Is that correct? I have written that the column vectors $[\mathbf{u}]$ and $[\mathbf{u}']$ also live in $\mathbb{R}^3$. Is that correct? (I feel like one must be wrong because how can you add a tuple to a column vector? they are different objects.) My main question is where does $\vec{\mathbf{u}}$ itself live? It can't be $\mathbb{R}^3$ because that would imply you could add $\vec{\mathbf{u}}$ to it's 3-tuple representation $\mathbf{u}$ which makes no sense to me. I often write $\vec{\mathbf{u}}\in\mathbb{E}^3$ and say that $\mathbb{E}^3$ is the manifold that is flat 3-space but I don't truly comprehend what that means (and I guess if $\vec{\mathbf{u}}$ is a velocity vector it would actually be in the tangent bundle of $\mathbb{E}^3$).

Similarly, for tome 2nd order tensor, $\mathbf{T}$ (which we could expand in terms of coordinates in some basis as $\mathbf{T}=T_{ij}\hat{\mathbf{e}}_i\otimes\hat{\mathbf{e}}_i$), I often denote the matrix representation of $\mathbf{T}$ (in some specified basis) as $T\in \mathbb{R}^{3\times 3}$. But where does $\mathbf{T}$ itself live? Would I say $\mathbf{T}\in\mathbb{E}^3\otimes\mathbb{E}^3$?

background: I have a BS in physics and current PhD student in engineering. I have a good understanding of cartesian tensor algebra/calculus. I have a poor/rudimentary understanding of differential geometry and calculus on manifolds. But enough that you don't need to totally avoid those subjects if they are needed to answer my question.

Answer 1

I think the core of your question is a very commonly-misunderstood subtlety, so I'll begin with a seemingly abstract example. Consider the vector space $V$ which consists of formal polynomials of degree $\leq 2$:

$$V := \big\{a+bx+cx^2 \ : \ a,b,c\in \mathbb R\big\}$$

This is an $\mathbb R$-vector space, as can be seen easily. If we choose a basis $\mathcal B:=\{1,x,x^2\}$, then we can associate an element $\vec v = a+b+cx^2$ with the $3$-tuple of real numbers which are its components with respect to $\mathcal B$, namely $$V\ni a+bx+cx^2 \iff \begin{bmatrix}a\\b\\c\end{bmatrix}_\mathcal B\in \mathbb R^3$$ I emphasize that $[\vdots]_\mathcal B\in \mathbb R^3$ are the components of the vector $\vec v\in V$ in the basis $\mathcal B$.

---

Now we choose another vector space - $\mathbb R^3$, the set of 3-tuples of real numbers. This is also an $\mathbb R$-vector space, and I will denote its elements using parentheses: $$\vec v = \pmatrix{a\\b\\c}$$ Here is the key point - $a,b,$ and $c$ are not to be understood as the components of $\vec v$ with respect to some unspecified basis. The underlying set is $\mathbb R^3$, and $\vec v = \pmatrix{a\\b\\c}$ is a genuine element of that space, not merely a collection of components.

If we choose a basis $\mathcal B = \left\{\pmatrix{1\\0\\0},\pmatrix{0\\1\\0},\pmatrix{0\\0\\1}\right\}$ then we also have that $$\mathbb R^3 \ni \pmatrix{a\\b\\c} \iff \begin{bmatrix}a\\b\\c\end{bmatrix}_\mathcal B \in \mathbb R^3$$ i.e. $a,b,c$ are also the components of $\vec v$ with respect to the particular basis $\mathcal B$. If we choose a different basis $\mathcal B' = \left\{\pmatrix{1\\1\\0},\pmatrix{1\\-1\\0},\pmatrix{0\\0\\1}\right\}$, then we would have that $$\mathbb R^3 \ni \pmatrix{a\\b\\c} \iff \begin{bmatrix}\frac{a+b}{2}\\\frac{a-b}{2}\\c\end{bmatrix}_{\mathcal B'} \in \mathbb R^3$$

In summary, for a generic $n$-dimensional $\mathbb R$-vector space $V$, we can pick a basis and then make an association between an element of $V$ and the list of its components, which is an element of $\mathbb R^n$. This is why every finite-dimensional $\mathbb R$-vector space is isomorphic to $\mathbb R^n$.

The confusion arises because a special case of an $n$-dimensional $\mathbb R$-vector space is $\mathbb R^n$ itself. An element of the vector space - which I denoted by $\pmatrix{\vdots}$, and which consists of an $n$-tuple of real numbers - can be associated to its components in some basis $\mathcal B$, which I denoted by $\begin{bmatrix}\vdots\end{bmatrix}_\mathcal B$ and which is also an $n$-tuple of real numbers.

In the polynomial example, it's easy to tell the difference between a vector and its components. However, when the vector space is also the $n$-tuples of real numbers, it's extremely easy to confuse a vector with its components, or to mistakenly believe that any $n$-tuple you write down must be with respect to some basis or another. I prefer to use the round and square brackets to distinguish between the two when ambiguity is possible, but this is not the norm even in educational literature. I am not sure why.

Answer 2

We should distinguish between vector spaces and the manifolds upon which they are tangent. A vector space is an abstract space where addition between the elements of the vector space is defined, as well as scalar multiplication. So column vectors with real parts such as the one you wrote form their own vector space, while 3-tuples do not; they are just ordered lists of numbers. As you said, since these two objects behave differently and can't mix, they do not rigorously live in the same space. The reason is that the 3-dimensional vectors, even though they are essentially 3-tuples themselves, they satisfy some additional requirements. For example, you can't define an inner product between 3-tuples when defined in terms of fields, but you can do so in the case of the vectors.

You can in fact define a mapping that takes you from the one to the other:

\begin{equation} \phi(\bf{u}) \rightarrow [\bf{u}] \end{equation}

and the inverse:

\begin{equation} \phi^{-1}([\bf{u}]) \rightarrow \bf{u} \end{equation}

which means that this mapping is an isomorphism. This is a mapping from a certain vector space $V$ to the space of all 3-tuples and vice-versa, so it doesn't tell us where they live quite yet.

Every 3-tuple is a set of (real) numbers which satisfy certain conditions in order to form a group, and in order to belong to $\mathbb{E}^{3}$ more specifically. Since they belong to this group, they can be represented by an appropriate manifold which is just going to be a flat 3-dimensional plane. So you could say that the 3-tuples "live" in the 3-dimensional Euclidean space. The column vectors will instead belong to the space of real 3-dimensional vectors $\mathbb{R}^{3}$ which is also a group and on top of that provides some bonus structure due to the scalar multiplication and vector addition properties. Both $\mathbb{E}^{3}$ and $\mathbb{R}^{3}$ have the exact same set of elements (all three-number-combinations of real numbers ordered in the same certain way) and since $\mathbb{E}^{3}$ is flat, the tangent vector space $\mathbb{R}^{3}$ ends up being identical in terms of its contents. Basically $\mathbb{R}^{3}$ can be seen as a space that implies both the manifold structure of $\mathbb{E}^{3}$ plus the additional vector space properties. In that sense they are practically equivalent and each vector can be directly mapped to a specific point on the Euclidean plane.

Similarly, the 2nd order tensor you mention will live inside a vector space that is appropriate for the representation you are using. If you use a matrix representation as you said, then $\bf{T}$ belongs to the vector space of $3 \times 3$ real matrices which is higher dimensional. There are now 9 elements to each "vector" quantity (essentially each matrix). However, you can't say it directly "lives" on $\mathbb{E}^{3} \otimes \mathbb{E}^{3}$ per se, since each point on the aforementioned manifold is simply 9-tuple. It is tangent on it because(again) the matrices which belong to this vector space form a group with identical elements as $\mathbb{E}^{3} \otimes \mathbb{E}^{3}$, but contain additional mathematical qualities.

The vector space $\mathbb{R}^{3 \times 3}$ therefore has to be tangent to two copies of the 3-dimensional Euclidean vector space that have been stacked so that the combinations between one basis element of the first plane with another of the second plane. Since you have three elements in each, you can see that there are 9 combinations in total. Visually, instead of stacking two Euclidean planes on top of each other, you took one of them and set it to be "perpendicular" to the other. In column vector notation:

\begin{equation} \bf{u} = \begin{bmatrix} u_{1} \\ u_{2} \\ u_{3} \\ \end{bmatrix} \, \in \mathbb{R}^{3} \end{equation}

\begin{equation} \bar{\bf{u}} = [\bar{u}_{1} \, \bar{u}_{2} \, \bar{u}_{3}] \, \in \bar{\mathbb{R}}^{3} \end{equation}

\begin{equation} \mathbb{R}^{3} \otimes \bar{\mathbb{R}}^{3} = \mathbb{R}^{3 \times 3} \end{equation}

\begin{equation} \bf{u} \otimes \bar{\bf{u}} = \begin{bmatrix} u_{1}\bar{u}_{1} & u_{1}\bar{u}_{2} & u_{1}\bar{u}_{3} \\ u_{2}\bar{u}_{1} & u_{2}\bar{u}_{2} & u_{2}\bar{u}_{3} \\ u_{3}\bar{u}_{1} & u_{3}\bar{u}_{2} & u_{3}\bar{u}_{3} \\ \end{bmatrix} \, \in \mathbb{R}^{3 \times 3} \end{equation}