State of a system in Quantum Mechanics and state vectors

Question

I'm taking a course in Quantum Mechanics and there is something I'm not being able to fully understand. On more elementary courses on Quantum Mechanics I've been told that the idea of Quantum Mechanics is that all information available about a particle is contained in a function $\psi : U\subset \mathbb{R}^3\to \mathbb{C}$ whose evolution is governed by Schrödinger's equation.

The way to understand $\psi$ is that $|\psi|^2$ is the probability density of the presence of the particle in a neighborhood of a point. In that case, we have hermitian operators which are linear operators on the space of those functions which are associated with dynamical properties of interest. Examples are the Hamiltonian $\hat{H}$ and the momentum operator $\hat{p}$.

In this picture of wave functions it is even possible to "deduce" what $\hat{p}$ should be from the requirement that it is the generator of spatial translations. Anyway despite the usual strangeness of associating a wave to a particle, things are quite clear here.

On the other hand, on the course I'm taking now after reviewing these ideas the teacher moved to a different approach. He started dealing with kets. So instead of working with $\psi$ he started working with $\left|\psi\right\rangle$. He said a ket is not the function $\psi$, but one abstract object associated with $\psi$ which he called the state vector.

When he introduced the space of kets, i.e. the state space $\mathcal{E}$, I've asked about it here and two things were said. Firstly, @ACuriousMind said that

The idea to stress here is that quantum mechanics does not necessarily take place as "wave mechanics" on $L^2(\mathbb{R}^3)$.

Secondly, @AlfredCentauri said that

In a certain sense, $\psi(x)$ is to $\left|\psi\right\rangle$ as $v^i$ is to $v$.

So that it seems the values of a wave function are just components of the state vector in the so called position basis.

Now, what I'm not getting yet is this idea of abstract states and state vectors. What really is a state and what is a state vector in Quantum Mechanics? Quantum Mechanics is not all about assigning wave behavior to matter with the property that the square of the absolute value of the wave function is a probability density? What are those state vectors in truth and how does it all relates to the usual introductory treatments of Quantum Mechanics?

Answer 1

A state is something that encodes our knowledge about the system.

And that's it.

There are many ways to encode a state in quantum mechanics. As a wavefunction ("Schrödinger representation"), as Fock momentum states ("Fock representation"), as a density matrix, as a ray in a Hilbert space, as a linear functional on the $C^*$-algebra of observables, as a point in a projective Hilbert space,...

Not all of these ways are always permissible. The "as a ray in a Hilbert space", "density matrix" and "functional on algebra of observables" are, to my knowledge, always possible, while e.g. the encoding as a wavefunction fails for quantum systems whose Hilbert space is finite-dimensional (e.g. qubits) because those don't have the usual position operator.

The representation as a density matrix generalizes to statistical quantum mechanics, that as a single ray in a Hilbert space or as a wavefunction does not. But whenever two of these descriptions are possible, they are, at the level of rigor of physicists at least, equivalent.

So the question "What really is a state in quantum mechanics?" doesn't have a single answer. Or any answer other than "it depends what you want to do with it".

But this should not surprise you, after all, it is the same in classical mechanics: You can have Newtonian or Lagrangian or Hamiltonian descriptions, and here a state is position and velocity, there a state is position and momentum, or even some generalized coordinates which have no direct physical meaning at all.

There is no truth to a state, be it classical or be it quantum, other than "it encodes all possible information about the physical system in a convenient way".

Answer 2

"What really is X" is a tricky question in physics, especially when dealing with such a fundamental and abstract theory as quantum mechanics. So I can give a mathematical definition:

Let $\mathcal{H}$ be a complex Hilbert space, that is, a complete vector space over $\mathbb{C}$ equipped with a positive-definite inner product $\langle\, \cdot\, |\, \cdot\, \rangle$. If you didn't understand that, don't fret too much. The "vector space with inner product" is really the important part. The elements of $\mathcal{H}$, which are usually called vectors, are also called kets in QM, and a ket is written $|\psi\rangle$. The $\psi$ is whatever you need to write to identify the vector; in the general case, $|\psi\rangle$ is an arbitrary vector.

As you know, for the case of a single particle moving in n-dimensional space, this is equivalent to the usual wavefunction formalism: the function is, as you say, the "components" of the state vector $|\psi\rangle$ in the position basis. So in this case, the Hilbert space formalism is just a more abstract way of saying the same thing. It's useful because it lets you speak of the state without specifying the basis. For example, we can take a wavefunction $\psi(x)$ in the position representation (that is, the position basis) and Fourier transform it to get the momentum space wavefunction. These functions correspond to the same ket expressed in different bases.

But not every system is a particle moving in space. In your words; QM is not all about assigning wave behavior to matter. For example, we might be interested in the particle's spin. Now the Hilbert space is finite dimensional; for a spin-1/2 particle we have $\mathcal{H} \cong \mathbb{C}^2$ (see the Feynman Lectures, Vol 3, for an introduction). Now states are two-component vectors; we can think of the basis vectors $(1,0)$ and $(0,1)$ as being spin-up and spin-down respectively. But we have to choose an axis for this; if we don't want to, then we have to use the abstract vectors, exactly like we do in $\mathbb{R}^n$.

This is just an example. The point is that the Hilbert space / ket picture lets you work in full abstraction; you don't need to specify your particular system. You can use the same notation and the same theorems no matter if you have one particle moving in 3 dimensions, or $m$ particles moving in $n$ dimensions each one having an arbitrary spin, while also considering their coupling to the quantum electromagnetic field, ... you get the iea.