How the position operator and the position basis are correctly defined? In Quantum Mechanics, if one deals with wave functions, the Hilbert space in question is $L^2(\mathbb{R}^n)$ for a particle in $n$-dimensions, and the position operator corresponding to the $i$-th coordinate is $X^i$ defined by
$$(X^i \psi)(a) = x^i(a) \psi(a), \quad \forall a\in \mathbb{R}^n.$$
Now, if we generalize things slightly, we have a state space $\mathcal{H}$ which is a Hilbert space whose state vectors $|\psi\rangle \in \mathcal{H}$ describes the possible states of the system. In that case, the position operator is usually defined by
$$\langle x | X |\psi \rangle = x \langle x |\psi \rangle,$$
but here comes a problem, this definition depends on the particular basis $\{|x\rangle : x \in \mathbb{R}\}$. But how those kets are defined? Well, they are defined exactly as the eigenvectors of the position operator.
So, to define the position operator, we define it in a certain basis. But to define the vectors of that basis, we define then as the eigenvectors of the position operator.
This is circular. After all, the definition of the operator is being done in terms of vectors which are defined in terms of the operator.
Also, in this picture I cannot use the definition in terms of wave functions. And the reason is exactly because the wavefunction is then defined as the function $\psi(x)= \langle x| \psi\rangle$, which again depends on the kets $|x\rangle$.
There should probably be a correctly way to define the position operator and the position basis without being circular.
In that case, how does one correctly and rigorously define the positon operator and the position basis? I believe it all comes down to defining the operator, because once we prove it is hermitian we get a basis which from the postulates is naturally the position basis.
 A: In QM the position operator and the Hilbert space of a particle are defined contextually: The Hilbert space is $L^2(\mathbb R^3, d^3x)$ and the operator position along $x_k$ is defined, in that space, as $(X_k\psi)(x):= x_k \psi(x)$ with the obvious domain.
You can adopt a more abstract viewpoint if you simultaneously define the momentum and the position operator satisfying CCR, exploiting the so-called Stone-von Neumann theorem. If the representation of the CCR is irreducible (and some technical requirements hold) then the Hilbert space is unitarily isomorphic to $L^2(\mathbb R^3, d^3x)$ and the position and momentum operators become the standard ones under the mentioned isomorphism.
Another more sophisticated approach relies upon the notion of (Mackey's) system of  imprimitivity, where the momentum operator is the generator of spatial translations of the three position momenta.
A: There is nothing circular.  We assume that the physical reality of a particle corresponds to states in some vector space.  At this point we don't know of a basis defined on this vector space because we haven't given it any structure.  We give it structure by assuming the position operator is a Hermition operator in this vector space with eigenvalues on the real line. Since we assumed it is Hermiton we know that its eigenvectors span the vector space and are orthogonal and thus they form a basis.  This is the first basis we define.

So, to define the position operator, we define it in a certain basis. 

This is what you are confused about.  We don't need to define the position operator with respect to another basis.  We can understand it's relationship to other another basis when we define a relationship between two different operators.
For example.  The most natural thing to do next is define the momentum operator as the generator of translation.  This will ensure it is Hermitian and again define a new basis on your vector space representing the physical state of a particle. This definition will also give you a relation between $P$ and $X$ allowing you derive commutation relations and how the eigenstates of one operator are represented by the eigenstates of another operator.  Now you can define the the "matrix" representation of the position operator in the position basis in which it is diagonal and in the momentum basis in which it is not.
