Where does ket vector live in rigged Hilbert space? Let's say rigged Hilbert space $(\mathcal{S},\mathcal{H},\mathcal{S}^{*})$ in Gelfand triple. Where would ket vector live in? Would it be $\mathcal{S}$? That is what I thought, but https://arxiv.org/abs/quant-ph/0502053 suggests that it is actually in anti-dual space of $\mathcal{S}$. (Page 3) So is this article right, and I am wrong?
 A: The paper is correct.  Note that in a rigged Hilbert space $(\mathcal S, \mathcal H, \mathcal S^*)$, we have that $\mathcal S \subset \mathcal H \subset \mathcal S^*$.  That is, $\mathcal S$ (the set of so-called test functions) is a subset of $\mathcal H$.  The only reason the rigged Hilbert space construction is required in the first place is that ket vectors corresponding to definite states of continuous observables like $X$ and $P$ aren't actually elements of $\mathcal H$, which means they're definitely not elements of a subset of $\mathcal H$.

I'll be more explicit to address your additional question.  Let $\mathcal H$ be $L^2(\mathbb R)$, which is (roughly) the space of square integrable functions $f:\mathbb R \rightarrow \mathbb C$.  Additionally, let $\mathcal S$ be the space of rapidly-decaying smooth functions, defined as follows:
$$\mathcal S := \left\{f \in C^\infty(\mathbb R) : \forall m,n\in\mathbb N, \sup_x\left| x^n \cdot \frac{d^mf}{dx^m }\right|<\infty \right\}$$
Essentially, $\mathcal S$ is the space of all functions to which you can apply the position and momentum operators as many times as you want, with the result still being bounded.  It's not difficult to show that $\mathcal S\subset L^2(\mathbb R)$.  It's less clear that $\mathcal S$ is dense in $L^2(\mathbb R)$, but that also happens to be true.
A linear functional on $\mathcal S$ is a map $\varphi:\mathcal S \rightarrow \mathbb C$ such that for all $f,g\in\mathcal S$ and $\lambda\in\mathbb C$,


*

*$\varphi(f + g) = \varphi(f)+\varphi(g)$

*$\varphi(\lambda f) = \lambda \varphi(f)$
An antilinear functional is the same, except $\phi(\lambda f)=\bar{\lambda} \phi(f)$ where the bar denotes complex conjugation.
The space of all linear functionals (which we will identify as bras) on $\mathcal S$ is called the dual space $S'$, while the set of all antilinear functionals (which we will identify as kets) on $\mathcal S$ is called the antidual space $S^*$.
Observe that any element $f\in\mathcal H$ can be identified with a linear functional $\varphi_f \equiv \langle f, \bullet \rangle$, which acts on some $g\in\mathcal S$ as follows:
$$\varphi_f(g) = \langle f,g\rangle$$
Furthermore, it can be identified with an antilinear functional $\phi_f \equiv \langle \bullet , f \rangle$ as well:
$$\phi_f(g) = \langle g,f\rangle$$
This implies at least that $\mathcal H \subseteq S'$ and $\mathcal H \subseteq S^*$.  However, the spaces $S'$ and $S^*$ are much bigger than $\mathcal H$.  
Momentum Eigenfunctions
Observe that the function $e^{ikx}$, which is not square integrable and thus not an element of $\mathcal H$, can be identified with the dual space element $\varphi_k$ and the antidual space element $\phi_k$ where for all $g\in \mathcal S$,
$$\varphi_k(g) = \int_{-\infty}^\infty \overline{e^{ikx}} g(x) dx = \int_{-\infty}^\infty e^{-ikx}g(x) dx$$
and
$$\phi_k(g) = \int_{-\infty}^\infty \overline{g(x)} e^{ikx} dx $$
You are more used to the bra-ket notation, in which $\varphi_k \equiv \langle k|$ and $\phi_k \equiv |k\rangle$.
Position Eigenfunctions
$S'$ and $S^*$ also contain elements that don't correspond to functions at all.  The Dirac delta distribution $\delta_a$ is deceptively simple - it just evaluates a function at $a$.  Define the linear functional $\delta_a$ and the antilinear functional $\delta^*_a$ simply as
$$\delta_a (g) = g(a)$$
$$\delta^*_a (g) = \overline{g(a)}$$
You are again more familiar with the notation $\delta_x \equiv \langle x|$ and $\delta^*_x \equiv |x\rangle$.


How are expressions like $\langle x|p\rangle, \langle x|y\rangle, \langle x|f\rangle$ defined? Is there something like a scalar product or norm defined on $S', S^*$?

One way to do it is as follows. Note that given $f,g\in\mathcal H$ and the associated linear (resp. antilinear) functionals $\varphi_g$ (resp. $\phi_f$), we have that
$$\varphi_g(f) = \phi_f(g) = \langle g,f\rangle$$
Because $\varphi_g \rightarrow \langle g|$ and $\phi_f \rightarrow |f\rangle$, we make the formal identification
$$\varphi_f(g) = \phi_f(g) \equiv \langle g | f \rangle$$
So $\langle g | f\rangle$ can be equivalently thought of either as (i) mapping $g$ to its linear functional and feeding it $f$, or (ii) mapping $f$ to its antilinear functional and feeding it $g$.
Extending our consideration to $\delta_x \rightarrow \langle x|$, we can say that $\langle x| f\rangle$ corresponds to feeding the state $f$ to the functional $\delta_x$ - which is to say, evaluating it at the point $x$:
$$\langle x | f\rangle = \delta_x(f) = f(x)$$
Note that there is no either-or equivalence here - since $\delta_x$ does not actually correspond to a state, it doesn't strictly make sense to imagine the reverse, in which we convert $f$ to a functional and feed it the (nonexistent) state corresponding to $\delta_x$.  Of course, if we're willing to accept that the "state" which corresponds to $\delta_a$ is the "delta function" $\delta(x-a)$, then you can think of it that way.
Next, note that 
$$\langle g|f\rangle =\int_{-\infty}^\infty \overline{g(x)} f(x)\ dx = \int_{-\infty}^\infty \langle g|x\rangle \langle x|f\rangle$$
we therefore make the formal identification with the identity operator:
$$\int_{-\infty}^\infty |x\rangle\langle x| \ dx \rightarrow \mathbb{I}$$
Note that taken literally (e.g. as a Lebesgue integral), the expression on the left does not make sense on its own.  However, following the formal symbolic rules we have developed, it acts as an identity operator on the space of kets (and on the space of bras).
That being the case, we must have that
$$\mathbb I \circ \mathbb I = \mathbb I$$
$$\int dx\ |x\rangle\langle x| \int dy\ |y\rangle\langle y| = \int dx \int dy |x\rangle \langle y| \cdot \langle x|y\rangle = \int dx |x\rangle\langle x|$$
which motivates the (again formal) definition $\langle x|y\rangle \equiv \delta(x-y)$.
Finally, we can perform the same manipulations on the "eigenfunctions" of the momentum operator.  One finds that (as above)
$$\langle k | f\rangle = \int_{-\infty}^\infty e^{-ikx} f(x) dx = \hat f(k)$$
So while $\langle x|f\rangle = f(x)$, we have $\langle k|f\rangle = \hat f(k)$ where $\hat f$ is the Fourier transform of $f$.
Inserting the identity operator,
$$\hat f(k) = \langle k|f\rangle = \int \langle k|x\rangle \langle x|f\rangle dx = \int \langle k|x\rangle f(x)\ dx$$
Comparing this to the above, we are motivated to define
$$\langle k|x\rangle = e^{-ikx}$$
$$\langle x|k\rangle = e^{ikx}$$
