The vacuum in quantum field theories: what is it? In Section 10.1 of his textbook Quantum Field Theory for Mathematicians, Ticciati writes

Assuming that the background field or classical source $j(x)$ is zero at space-time infinity, the presence of $j$ will not affect the physical vacuum, $|0\rangle _P$.

(The context is $\phi ^4$ theory.)
First of all, what is the physical vacuum?  My first thought at properly defining a vacuum state would be:
Definition 1:  A quantum state is said to be a vacuum state iff the expectation value of the Hamiltonian in this theory is a local minimum (the Hamiltonian of course being part of the data that defines the theory).
Is this the correct notion of what it means to be the "physical vacuum" in a given theory?  If so, two questions immediately come to mind:
(1) To what degree is the vacuum unique?  I've heard many times that we have so-called "degenerate vacuum".  Presumably this means there is some sort of non-uniqueness going on.
(2) Is a physical vacuum necessarily Poincaré invariant?  (In relativistic quantum mechanics, the projective Hilbert space that is the space of states comes with an action of the Poincaré group that preserves probabilities, so it makes sense to talk about whether states are invariant or not.)  If, with this definition, a physical vacuum is not necessarily Poincaré invariant, then we better change our definition to include this, that is:
Definition 2:  A quantum state is said to be a vacuum state iff it is Poincaré-invariant and the expectation value of the Hamiltonian in this theory is a local minimum.
Then,
(3) With this alternative definition, to what degree is the vacuum unique?
Secondly, given the appropriate definition of "physical vacuum", how would $j$ affect this state in the case that it did not vanish at space-time infinity?
 A: You're right that the vacuum is the state that minimizes the energy. In the classical limit this is easy to do. Let's take $\phi^4$ theory for example. Then the Hamiltonian is $\dot{\phi}^2/2+(\nabla \phi)^2/2+\lambda \phi^4/4!$. The lowest energy configuration is thus the one where $\phi$ is constant sitting at $\phi=0$, the bottom of the potential.
However, we expect that quantum corrections will modify the definition of the vacuum state. For example, consider the quantum mechanics of a particle in a double well potential. Classically the particle wants to sit in one of the wells, but we know that the true vacuum is an even linear combination of the two classical vacua, due to tunneling, and that the energy of the odd combination must be lifted. In fact this occurs at exponentially small couplings, due to nonperturbative instanton contributions.
In general one needs to minimize the quantum effective potential, see Coleman's book. 
The vacuum does not have to be unique. One stupid example is the case of spontaneous symmetry breaking, say with a mexican hat potential. In this case there are a circle's worth of vacua. The reason this is a stupid example is that the physics in all the vacua is the same, since they are related by the nonlinearly realized global symmetry.
A better example is the case of certain supersymmetric theories, where one finds a rich moduli space of classical vacua. For example, for $N=1$ gauge theories in the absence of Fayet-Iliopolous terms, this space is parameterized by the set of independent holomorphic gauge invariant monomials in the scalar fields. Unlike the case of spontaneous symmetry breaking, these are not related by global symmetries, so they are truly nondegenerate vacua with different physics. This situation usually carries over to the quantum theory as well, due to the nonrenormalization of the superpotential. However, the Kahler potential can get quantum corrections, so the metric on the moduli space is modified quantumly.
The vacuum is not always Poincare invariant. In the case of $\phi^4$ theory it is because the expectation value of $\phi$ is homogeneous and static, namely $\langle \phi\rangle=0$. A counterexample is the case of a string embedded through spacetime. This breaks the transverse translation symmetries, which leads to $d-2$ transverse Goldstone bosons (the $X^i$'s of light-cone gauge in string theory.) You have to be careful about counting the Goldstones for spontaneously broken Poincare invariance, see Low and Manohar, http://arxiv.org/abs/hep-th/0110285. Even if the vacuum is Poincare invariant it does not have to be unique, as we can see from the above example from supersymmetry.
I'm not sure what would happen if you had a source that didn't vanish at infinity, wouldn't your energy diverge? This seems bad.
