From what I currently understand given a general state vector $|\psi\rangle$ the wave function: $$\psi(x) = \langle x|\psi\rangle$$ represent the vector $|\psi\rangle$ in the base of the eigenvalues of the position operator. Similarly the wave function $$\psi(p)=\langle p|\psi\rangle$$ represent the same vector but in the base of momentum. In practice we can think of wave functions as column vectors with an infinite number of entries, one for every real number.

So when we write $|\psi\rangle$ do we mean to represent the abstract vector $|\psi\rangle$ without referring to a specific base? Why do we do this? In friendly 3D linear algebra we almost always think of vectors in the context of a specific representation of them in some base. Wouldn't be easier to always represent states vectors in some specific base, so as wave functions? I am saying this because using this double way of representing vectors sometimes tends to make things confusing; for example: in QM lectures happens often that a certain operator is described as acting on ket vectors: $$A|\psi\rangle$$ and then after a bit the same operator, without any further explanation, is shown as acting on functions: $$A\psi(x)$$ But there are some operations that make sense only if applied on functions and not on ket vectors. Why do we represent things in such a way? Why don't we only use wave function representation of vectors in some specific base?