The formal notion you're looking for is that of an affine space. You have already linked a question (2) to which I've given a fairly thorough technical answer, so I'll avoid the formalism here.
Yes, you are essentially correct. Displacement vectors can be identified with the "difference" between two points, and are fundamentally baked-in to the structure of an affine space. If you then choose an origin point, you can define the position vector corresponding to any individual point as the displacement vector between that point and your chosen origin.
Therefore, displacement vectors are ever-so-slightly more fundamental in the sense that position vectors require you to make an additional choice of origin from the set of points in your affine space, while the displacements themselves require no such additional structure.
To answer your follow up question, the term "vector" shows up in a wide variety of distinct but related contexts. If you want a formal definition, then I would say that that a displacement vector is an element of a vector space which is incorporated into an affine space.
More generally, a vector is simply an element of a vector space. This is perhaps among the least satisfying answers imaginable, but there's really no other way to say it. The vectors which arise in differential geometry (i.e. tangent vectors to a manifold) are completely different beasts to the vectors which we've discussed here.
Lastly, in physics we often define a vector by how its components transform under rotations (or Lorentz transformations, in the case of 4-vectors). You can take that as the defining characteristic, but this can be overly broad$^\dagger$ and requires a careful discussion of symmetry transformations to treat properly.
$^\dagger$For instance, in SR the gamma matrices $\gamma^\mu$ transform like 4-vectors despite not actually being 4-vectors.