Trace as integral Consider a system of two entangled harmonic oscillators. The normalised ground state is denoted by $\psi_0(x_1,x_2)$.
I've been taught that a density matrix is constructed as $\rho = \left|\psi\rangle\langle\psi\right|$, so in this basis: $$\rho = \psi_0(x_1,x_2) \psi_0^*(x_1',x_2') \left|x_1,x_2\rangle\langle x_1',x_2'\right|$$ The reduced density matrix of the second oscillator is then: $$\rho_2(x_2,x_2') = \psi_0(x_1,x_2) \psi_0^*(x_1,x_2') \left|x_2\rangle\langle x_2'\right|$$
However, in a paper I've come across this reduced density matrix is written: $$\rho_2(x_2,x_2') = \int_{-\infty}^{\infty} dx_1 \psi_0(x_1,x_2) \psi_0^*(x_1,x_2')$$
I'm not familiar with the Trace as an integral, though I can sort of see how it would work for continuous variables. Clearly there's also a difference in notation, since this last formula has no bras or kets.
I was wondering if someone could explain the difference, or maybe give some sort of overview.
 A: If the eigienvalues form a continuous spectrum, like the eigenvalues of $x$, then states must be normalized to a dirac delta, 
$$ \left\langle x \right| x' \rangle = \delta(x-x') $$
The trace of an operator is the sum of the diagonal elements, or if the basis is continuous, it becomes an integral
\begin{eqnarray*}
\mathrm{Tr}\left(\left|\phi\right\rangle\left\langle \psi \right|\right) &=& \int_{-\infty}^{\infty}\mathrm{d}q\,\left\langle q\right|\phi\rangle\left\langle \psi \right|q\rangle\\
&=&  \int_{-\infty}^{\infty}\mathrm{d}q\,\phi(x)\psi^*(x') \left\langle q\left|x\left\rangle\right\langle x'\right|q\right\rangle\\
&=& \int_{-\infty}^{\infty}\mathrm{d}q\,\phi(x)\psi^*(x') \delta(q-x)\delta(q-x')
\end{eqnarray*}
This last line is only nozero if $x=x'=q$, so you can choose whichevr label you like for the integration variable.
$$\mathrm{Tr}\left(\left|\phi\right\rangle\left\langle \psi \right|\right) = 
\int_{-\infty}^{\infty}\mathrm{d}x\,\phi(x)\psi^*(x)$$ 
As for as the difference between the kets and no kets, if the kets are there, it is the operator, ready to act on some vector. If the kets are not there, you have just the $x_1,x_2$ matrix  element. It's technically not the operator, but since the elements are just the values of some function of $x_1,x_2$, writing the element this way tells you all of the information you need. 
For example, you could write the 2x2 identity matrix as $$\mathbb{I} = \pmatrix{1&0\\0&0} + \pmatrix{0&0\\0&1}$$
You could just as easily say
$$\mathbb{I}_{i,j} = \begin{cases}1& \mathrm{if}\;i=j\\ 0 & \mathrm{otherwise}\end{cases}$$
and convey the same information.
A: For the benefit of possibly you and others, I'd first like to note that given a quantum system with Hilbert space $\mathcal H$, the density matrix $\rho$ of this system is not necessarily of the form $\rho = |\psi\rangle\langle\psi|$.  When the density matrix of the system has this special form, it is said to be in a pure state.
Ground state density matrix.
If $|\psi_0\rangle$ is the ground state of the system, then when the system is in its ground state, the density matrix of the system is
\begin{align}
  \rho = |\psi_0\rangle\langle\psi_0|. \tag{1}
\end{align}
Note, in particular, that the density matrix is an operator on $\mathcal H_1\otimes\mathcal H_2$, the Hilbert space of the composite system.  
Now, let $\{|x\rangle\}$ denote the standard position basis for a single harmonic oscillator, then the (tensor) product states $|x_1\rangle|x_2\rangle$ yield a basis for the system of two oscillators.  If $|\psi\rangle$ is a state of the composite system, then it can be expanded in the position basis as follows:
\begin{align}
  |\psi\rangle = \int dx_1\int dx_2\, \psi(x_1, x_2)|x_1\rangle|x_2\rangle,
\end{align}
where the function $\psi(x_1, x_2)$ is the position space representation of the state $|\psi\rangle$.  In particular, this can be done for the ground state;
\begin{align}
  |\psi_0\rangle = \int dx_1\int dx_2\, \psi_0(x_1, x_2)|x_1\rangle|x_2\rangle.
\end{align}
Plugging this into $(1)$, we find that the density matrix for the composite system in its ground state is
\begin{align}
  \rho = \int dx_1 \int dx_2 \int dx_1'\int dx_2' \,\psi_0(x_1, x_2)\psi_0^*(x_1', x_2')|x_1\rangle |x_2\rangle\langle x_1'|\langle x_2'| \tag{2}
\end{align}
Note that the matrix elements of this density matrix in the position basis are given by
\begin{align}
  \rho(x_1, x_2, x_1', x_2') = \langle x_1|\langle x_2|\rho|x_1'\rangle|x_2'\rangle = \psi_0(x_1, x_2)\psi_0^*(x_1', x_2')
\end{align}
where the first equality is just notation, and the second follows from Dirac-orthonormality of the position basis which I leave to you to show.
Subsystem density matrix - partial trace.
If the composite system is in its ground state, then as you indicate, we can obtain the density matrix of the second oscillator by taking a partial trace over the first Hilbert space factor $\mathcal H_1$.  To do this, we recall that the partial trace is linear and has the property that for operators $O_1$ and $O_2$ on the Hilbert spaces $\mathcal H_1$ and $\mathcal H_2$ respectively
\begin{align}
  \mathrm{tr}_1(O_1\otimes O_2) = (\mathrm{tr} O_1)O_2
\end{align}
Moreover, we note that
\begin{align}
  |x_1\rangle |x_2\rangle\langle x_1'|\langle x_2'| = |x_1\rangle\langle x_1'|\otimes |x_2\rangle \langle x_2'|
\end{align}
so that
\begin{align}
  \mathrm{tr}_1(|x_1\rangle |x_2\rangle\langle x_1'|\langle x_2'|)
&= \mathrm{tr}(|x_1\rangle\langle x_1'|)|x_2\rangle \langle x_2'| \\
&= \delta(x_1 - x_1') |x_2\rangle \langle x_2'|
\end{align}
and therefore the density matrix for subsystem 2 when the composite system is in its ground state is
\begin{align}
  \rho_2 = \int dx_1 \int dx_2 \int dx_2' \,\psi_0(x_1, x_2)\psi_0^*(x_1, x_2') |x_2\rangle|\langle x_2'|
\end{align}
while its matrix elements in the position basis are readily compute as
\begin{align}
  \rho_2(x_2, x_2') = \langle x_2|\rho_2|x_2'\rangle = \int dx_1 \psi_0(x_1 x_2)\psi_0^*(x_1, x_2')
\end{align}
