$\renewcommand{\Im}{\operatorname{Im}}$Let's dissect the question a bit. First, the periodic BCs on the free boson on a torus correspond to (R,R). But of course, no one stops you from imposing (R,NS), (NS,R) and (NS,NS) BCs and compute the partition function. The physical meaning of this is having turned on a background $\mathbb{Z}_2$ gauge field along either or along both cycles of the torus. So to rephrase your question, you essentially ask the following:
We know that $$Z_{\text{(R,R)}}[\tau] = \frac{1}{\sqrt{\Im(\tau)}}\frac{1}{\left|\eta(\tau)\right|^2}.\tag{0.1}\label{1}$$
What is $Z_{(\bullet,\circ)}[\tau]$ with $(\bullet,\circ)\in\{\text{(R,NS), (NS,R), (NS,NS)}\}?$
There are two ways to answer, and both of them use the following fact of life:
The partition function of the non-compact scalar on a torus
$\mathbb{T}^2_\tau:=\mathbb{C}/\left(\mathbb{Z}\oplus\tau\mathbb{Z}\right)$,
with $\tau\in\mathbb{H}$ (the upper half plane), can be read from the
partition function of the compact scalar by sending the
compactification radius to infinity, $R\to\infty$.
Way 1
The simplest way to obtain the answer is as follows.
Well, it suffices to go look at the compactified case and stare at the orbifold computation. For example stare at equation (8.24) in Ginsparg's notes. You will see that it is only the (R,R) sector that contributes to the $R$ dependence. Therefore, the (R,NS), (NS,R) and (NS,NS) are identical in the uncompactified case (when you sent $R\to\infty$). So we have
\begin{align}
Z_\text{(R,NS)}[\tau] &= \left|\frac{2\eta(\tau)}{\vartheta_2(\tau)}\right| \tag{1.1}\label{1.1} \\
Z_\text{(NS,R)}[\tau] &= \left|\frac{\eta(\tau)}{\vartheta_4(\tau)}\right| \tag{1.2}\label{1.2}\\
Z_\text{(NS,NS)}[\tau] &= \left|\frac{\eta(\tau)}{\vartheta_3(\tau)}\right|. \tag{1.3}\label{1.3}
\end{align}
Way 2
Another way, is to do the computation from scratch. Namely, go back to the path integral and compute, say in the (R,R) case
$$Z_\text{(R,R)}[\tau] = \frac{\operatorname{vol}(\text{zero-modes})}{\sqrt{\operatorname{det}_\text{(R,R)}'\!\big(\partial\bar\partial\big)}}.\tag{2.1}\label{2.1}$$
Up to factors of $2$ and $\pi$, you can then see the following:
$$\operatorname{vol}(\text{zero-modes}) = \sqrt{\Im(\tau)}\tag{2.2}\label{2.2}$$
and the non-zero eigenvalues of $\partial\bar\partial$ on a torus with (R,R) BCs are simply
$$\lambda^\text{(R,R)}_{n,m} = \frac{1}{\Im(\tau)^2}\left|n+\tau m\right|^2, \qquad (n,m)\in\mathbb{Z}^2\setminus(0,0),$$
giving
$$\operatorname{det}'_\text{(R,R)}(\partial\bar\partial) = \Im(\tau)^2\left|\eta(\tau)\right|^4.\tag{2.3}\label{2.3}$$
Altogether plugging \eqref{2.2} and \eqref{2.3} in \eqref{2.1}, gives \eqref{1}.
Now, for the other boundary conditions, all you have to do is observe that an NS BC along either cycle shifts either (or both) $n$ or $m$ by $\frac{1}{2}$, therefore, e.g. for (R,NS) BCs you have
$$\lambda^\text{(R,NS)}_{n,m} = \frac{1}{\Im(\tau)}\left| n+\tau\left(m+\frac{1}{2}\right) \right|^2, \qquad n,m\in\mathbb{Z}^2.$$
Note that now you don't have a zero-mode anymore. So all you have to do now is compute the determinant of $\partial\bar\partial$ with these boundary conditions and find
$$Z_{(\bullet,\circ)}[\tau] = \frac{1}{\sqrt{\det_{(\bullet,\circ)}(\partial\bar\partial)}}.$$
Doing this should land you on \eqref{1.1}-\eqref{1.3}.
