Interpretation of the nonuniqueness of vacuum of QFT in flat spacetime for a given inertial observer; No Lorentz transformation; No accelerated motion 
*

*Consider an inertial observer in flat spacetime with a choice of coordinates $(t,{\vec x})$. This observer can expand a quantum field $\hat{\phi}$ in more than one complete set of orthonormal modes. Let us consider two such choices made by the observer in question.


*In terms of modes $\{f_i\}$, the quantum field is expanded as
$$\hat{\phi}(t,{\vec x})=\sum_i\left(\hat{a}_if_i+\hat{a}_i^\dagger f_i^*\right)$$
$$[a_j,a_j^\dagger]=\delta_{ij}, [a_i,a_j]=[a_i^\dagger, a_j^\dagger]=0.$$

*

*Similarly, in terms of modes $\{g_i\}$, the same field $\hat{\phi}(t,{\vec x})$ is expanded as
$$\hat{\phi}(t,{\vec x})=\sum_i\left(\hat{b}_ig_i+\hat{b}_i^\dagger g_i^*\right)$$
$$[b_j,b_j^\dagger]=\delta_{ij}, [b_i,b_j]=[b_i^\dagger,b_j^\dagger]=0.$$

*

*It can be shown that the basis modes are related by the Bogoliubov transformation.


*Further, it can be shown that the vacuum defined by $a_i|0_f\rangle=0$ is different from the vacuum defined by $b_i|0_g\rangle=0$. In particular, though $$\langle 0_f|a_i^\dagger a_i|0_f\rangle=0,$$ $$\langle 0_f|b_i^\dagger b_i|0_f\rangle\neq 0.$$


*Does it mean that even for a given inertial observer the notion of vacuum is ambiguous? If more than one choice of annihilation operators (e.g. $a_i$ and $b_i$, above) is possible, how can we define a unique vacuum? Is there at all a preferred choice of basis modes (and thus, a preferred choice of creation and annihilation operators) and a preferred vacuum?
Please note that I am (i) neither talking about changing from one inertial frame to another (Lorentz transformation), (ii) nor about curved spacetime or accelerated observers.
I asked a related question here, but I did not receive a satisfactory answer probably because the question was not focussed and poorly worded.
 A: Valter Moretti gave a great answer from the perspective of rigorous mathematics. As a supplement, I'll add an answer from the less rigorous perspective of for-all-practical-purposes (FAPP) physics. The context of the question is flat spacetime with a single observer, but generality is one of the keys to insight, so I'll start with a broader perspective and then apply it to the question.
 The FAPP approach 

*

*The energy operator is the operator that generates time-translations.


*In quantum field theory, the vacuum state is the state of lowest energy. It can be non-unique due to things like spontaneous symmetry breaking, but that's a different subject. In many models, including models of free fields, it's unique — after we specify the energy operator.


*Proper time is observer-dependent, and it's a local concept (applicable only in a neighborhood of that observer), not a global concept.
People often point out that the definition 1 only makes sense in a spacetime that has time-translation symmetry, like flat spacetime. But here's the FAPP theme: In the neighborhood of any given observer, we can always define an effective Hamiltonian $H=\int_R T^{00}$, where $T^{ab}$ is the energy-momentum tensor, $R$ is a spatial neighborhood of the observer, and the coordinate system is such that the "time" coordinate (index $0$) agrees with the observer's proper time. The operator $H$ works just fine as a generator of time-translations in a neighborhood of that observer, so we can use it in the definition 2: any state $|0\rangle$ that minimizes the expectation value of $H$ qualifies as a vacuum state locally, for that observer.
(Warning: We cannot let the local integration region $R$ be too small — not microscopic — because the energy density $T^{00}$ is not bounded below in relativistic QFT. I cited some references about this in an answer to another question. In this answer, I'm assuming that the spacetime curvature and the observer's acceleration are both mild enough so that $R$ can be large enough to make that effect unimportant.)
Particles are defined with respect to the vacuum state. But the vacuum state is the state that minimizes the energy, and the energy operator is observer-dependent (and typically only defined locally, as the generator of that observer's proper time translations locally), so particles are observer-dependent.
By the way, the expression $\int_R T^{00}$ for the (local) energy operator illustrates the important idea of a splittable symmetry — a symmetry that can be applied locally, to only part of a system. I described the significance of splittable symmetries in another answer. This is the kind of "time translation symmetry" that matters when we're talking about a given observer, because observers are localized.
 Application to inertial observers in flat spacetime 
In flat spacetime, thanks to time-translation symmetry, the vacuum state can be defined globally. And thanks to Poincaré symmetry, the vacuum state is the same for all inertial observers, even though different inertial observers have different energy operators. Therefore, all inertial observers in flat spacetime agree about the definition of "particle."
The question asks about a single inertial observer in flat spacetime. The answer is that physical particles are defined with respect to the lowest-energy state, and the energy operator is uniquely defined, so the lowest-energy state is also uniquely defined — ignoring things like spontaneous symmetry breaking, which is beside the point here.
 Accelerating observer in flat spacetime: the Unruh effect 
By the way, we can apply this same approach to the famous Unruh effect. The Unruh effect considers a uniformly accelerating observer in flat spacetime. For a uniformly accelerating observer, we can use the operator that generates boosts as the operator $K$ that generates translations of that observer's proper time. (The boost operator is globally defined, but that's not important. The important thing is that it's defined in a neighborhood of the observer, where it generates translations of that observer's proper time.) Locally, we can define a set of states that minimizes the expectation value of a local version of $K$. All of these states look the same locally, to that observer, and none of these states agree with the Minkowski vacuum state. That's why a uniformly accelerating observer "sees" particles in the Minkowski vacuum, and conversely why inertial observers "see" particles in the accelerating observer's vacuum. This is just a straightforward application of the definitions 1,2,3, applied locally FAPP.
 The Hawking effect 
We can also apply this approach to the even-more-famous Hawking effect. Hopefully that's pretty obvious now, so I won't ramble on about it.
 A useless definition 
Every once in a while, you might encounter a book/paper that defines the vacuum state as the state annihilated by a set of mutually commuting annihilation operators — operators $a_n$ whose commutator with their adoints is $[a_n,a_m^\dagger]=\delta_{nm}$. Authors are free to make whatever definitions they want, because language is arbitrary, but it's important to understand that the annihilation-operator-based definition is generally completely different than the lowest-energy definition that is standard in the quantum field theory literature. To see just how arbitrary the annihilation-operator-based definition is, suppose that $|\varnothing\rangle$ is the state satisfying $a_n|\varnothing\rangle=0$ for all $a_n$, and let $|\psi\rangle$ be absolutely any other state. A unitary operator $U$ satisfying $U|\varnothing\rangle=|\psi\rangle$ always exists, and the operators $b_n\equiv U^{-1}a_n U$ satisfy the same type of commutation relations as the original operators: $[b_n,b_m^\dagger]=\delta_{nm}$. Therefore, according to the annihilation-operator-based definition, every state in the Hilbert space would be a vacuum state. That's not a very useful definition, but it's not illegal, either. It's just different... and useless.
 A similar but more useful definition 
Given a preferred time coordinate, such as one that agrees locally with a given user's proper time, we can define positive frequency and negative frequency. And given any time-dependent operator in the Heisenberg picture, we can define its positive- and negative-frequency parts. In free field theories, the standard set of creation and annihilation operators are defined to be the negative- and posititive-frequency (respectively) parts of the field operators. Those annihilation operators annihilate the lowest-energy state, because the positive-frequency part of any time-dependent operator acts as an energy-lowering operator. When applied to a state that already has the lowest possible energy, an energy-lowering operator annihilates it.
So if we define the vacuum state as the state to be the state annihilated by the negative-energy parts of all time-dependent operators, then this is equivalent to defining the vacuum state as the state of lowest energy. In free field theory, these operators are called annihilation operators, but notice that these are not just any old set of annihilation operators. They're defined with respect to a given energy operator (generator of time translations), and that's the key to answering the question.
A: First of all, a state of the type you consider is said to be Gaussian or quasi-free. These states are completely defined by their two-point function $\omega(t_x,x,t_y,y)$  -- which is supposed to define a positive bi-distribution and whose antisymmetric part is fixed by the standard CCR -- using Wick's rule to compute the even $n$-functions (the odd ones being $0$). I stress that the two-point function defines a Gaussian state intrinsically without referring to an explicit Fock space structure that is an a posteriori structure defined up to unitary transformations.
In the case of the two-point function of a Gaussian state,
(a) the Hilbert space is a Fock space,
(b) the operatorial representation of $\phi$ is the standard one,
(c) $\omega(t_x,x,t_y,y)$ is exactly “expectation value” of the vacuum vector of the Fock space as the notation suggest
$$\omega(t_x,x,t_y,y) = \langle 0| \phi(t_x,x)\phi(t_y,y)\rangle |0\rangle\:.$$
A uniqueness result arises as soon as one requires that the two-point function (and thus the vacuum) has good properties with respect to the time translation referred to the considered reference frame and a pair of other physically meaningful requirements.
(1) The two point function is invariant under time displacements
$$\omega(t_x+T,x,t_y+T,y) = \omega(t_x,x,t_y,y) \quad \mbox{for every $T\in \mathbb{R}$}$$
(2) In the Fock space the time translation is implemented by an Hamiltonian operator $H$ such that its spectrum is non-negative and $|0\rangle$ is the unique zero eigenvector of $H$ up to phases.
(3) The representation of the algebra of fields is irreducible.
There is only one two-point function satisfying the said hypotheses.
Actually (1) and a physically natural continuity requirement  imply the existence of the generator $H$, so that (2) matters only concerning the spectrum and the eigenvalue property.
In Minkowski spacetime, when switching on the Poincaré invariance, this state must coincide with the analog vacuum state of every other inertial frame.
It is worth stressing that, dropping the request of dealing with Gaussian states, the picture dramatically changes and the result is no longer valid in general.
The appropriate setup to prove the statement above is the algebraic formulation of quantum field theory. The mentioned result was established by B.Kay [1]   in a quite general fashion, e.g., in curved spacetime  and other contexts (we used it for instance to define ground states on the light-like boundary of asymptotically flat spacetimes [2] and in the construction of the Unruh state in the Kruskal spacetime [3]),
[1] B. S. Kay, A uniqueness result in the Segal-Weinless approach to linear Bose fields,J. Math. Phys.20, 1712 (1979)
[2] V.Moretti:  Uniqueness theorems for BMS-invariant states of scalar QFT on the null boundary of asymptotically flat spacetimes and bulk-boundary observable algebra correspondence Commun. Math. Phys. 268, 727 (2006).
[3] C. Dappiaggi,  V.Moretti, and N. Pinamonti: Rigorous construction and Hadamard property of the Unruh state in Schwarzschild spacetime.
Adv. Theor. Math. Phys. 15, vol 2, 355-448 (2011)
A: Others have already given a deep discussion of the details of the problem, but I wanted to add a simpler approach to your doubt.
The issue with using the definition of vacuum as the state which is destroyed by all annihilation operators is that this is not true for any set of operators, but specifically for the set of operators which diagonalizes the Hamiltonian.
In the diagonal form $\hat{H}$ can be written as
\begin{align}
\hat{H} = \sum_n E_n \hat{a}_n^{\dagger} \hat{a}_n,
\end{align}
where $E_n>0$.
It's straightforward to check that state $\vert \psi\rangle$ which satisfies $\hat{a}_n \vert \psi\rangle= 0$ for all $\hat{a}_n$ is the state with minimum energy (which is the fundamental property of the vacuum).
Note that because of this there is no ambiguity for the vacuum state because the set of operators which diagonalize the Hamiltonian is unique (given no degeneracy is present), apart from a global phase.
