Collapse of the wave function in non-discrete systems Let's consider a discrete quantum system such as the harmonic oscillator or the hydrogen atom. According to quantum mechanics, when a measurement of the energy of the system is made the wave function collapses to one of the eigenfunctions of the hamiltonian operator. But if we consider instead a non-discrete system, such as a free particle, the eigenfunctions of the hamiltonian operator are non-normalizable functions, so the wave function can't collapse to one of those. What does quantum mechanics says in this case? Can it be determined, knowing the result of the measurement, what is the new wave function (like we can in a discrete system using accurate-enough instruments)?
 A: It depends on the type of measurement you perform.  The idealized case is described by the PVM (projector valued measure) uniquely  associated to the observable viewed as a selfadjoint operator through the spectral theorem.
The PVM is a  collection of orthogonal projectors $P_E$, where $E$ is a Borel set of the real axis, typically a finite interval, defined in the practice by the precision of the instrument.  This collection of projectors satisfy some mathematical properties similar to those of a positive measure.
If the initial state is represented by $\psi$ and the outcome is $E$, the post-measurement state  is always described by the vector  $P_E\psi \neq 0$ up to normalisation.
Here $||P_E\psi||^2$ is the probability to obtain the outcome $E$ when the initial state is represented by the normalized vector $\psi$.
All that  is nothing but the Luders-von Neumann postulate.
If the spectrum is continuous, single points $E=\{\lambda \}$ have automatically zero projector $P_E=0$, so that "non-normalizable vectors" cannot be produced this way.
For instance, for the position operator, if the position measurement produced the outcome $E=[a,b]$, the corresponding projector is
$$(P_E \psi)(x) := \chi_E(x) \psi(x)$$
where the function $\chi_E$ is zero outside $E$ and $1$ in $E$.(Notice that if $E$ is a single point, the associated projector is zero since single points have zero Lebesgue measure.)
It is worth stressing that  this description is valid also when the spectrum is a point spectrum. In that case single points (eigenvalues) have non-zero projectors: the ones onto eigenspaces.
A more realistic description is provided by a POVM (positive operator valued measure) and its decomposition (it is not unique) in terms of Kraus operators, but also this description does not give rise to non-normalized state vectors.
