closed as unclear what you're asking by ACuriousMind♦, user36790, Gert, Bill N, Danu Apr 3 '16 at 11:49
Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
In brief: phase space is not made into a vector space because that additional structure provides no benefit; quantum mechanics uses a Hilbert space because that additional structure does provide benefits.
Any time you relate a mathematical structure to a physical concept, you need to ask how useful that relation is. The mathematical structure will have various properties, so the question is: does the physical concept embody all of those properties, and are those properties enough to describe the physics?
In this case, the main mathematical structure you're asking about is the vector space, which has two key properties: vector addition, and multiplication by a scalar. In quantum mechanics, there's a physical concept of interference between two particles. The mathematical structure of the vector space is able to represent this as addition of the two vectors (wavefunctions) representing the particles. We also have the concept of superposition of states for a single particle, which is also represented by the addition of those states. But just adding the two possible states gives you a wavefunction where the probability of finding the particle in that state is greater than one. So you need to be able to multiply the possible states by a scalar. Moreover, there might be just a tiny possibility of finding the particle in one of those states, so you'd like to mix in just a small amount of that vector; you'd multiply it by some small scalar.
These things are just arguments suggesting that the vector space gives you a reasonable way of representing the physical situation, and getting rid of any of the features of the vector space mean that you get rid of some of its ability to describe physics. So we probably don't want a simpler mathematical structure, and there's no obvious reason to have a more complicated mathematical structure.
Okay, now let's turn to the phase space. For simplicity take the usual example of a single particle in one-dimensional motion, so the phase space is two dimensional. Recall that one of those dimensions represents the physical concept of position, while the other represents the physical concept of momentum. You actually could make a vector space out of these two dimensions; just define a point to be a vector in terms of its coordinates, define addition of two points by addition of their coordinates, and define scalar multiplication by multiplication of its coordinates. That's a perfectly valid mathematical structure.
But now we have to ask our question. What does this fancy mathematical structure get you? Are vector addition and scalar multiplication useful expressions of some physical concept? What does it mean to simultaneously multiply your position and momentum by 2, for example? What does it mean to add the particle having position 1 and momentum 0 to the particle having position 0 and momentum 1? That's just another state, probably with a different energy. Basically, there won't typically be any useful physical interpretation of these operations.
Pretty generally (though not absolutely always), it's not useful in physics to add things of different types. In this case, position and momentum are quite different concepts, so it's not at all clear why you should add them together. From a mathematical perspective, you could define it. But from a physical perspective, it's not clear why you should.
 As the OP said, quantum mechanics actually uses a Hilbert space, which is a vector space with another important property. Not only is it a vector space, but it also has an inner product defined on it; you can take two vectors and get a scalar out. (There are a few more technical details, but that's the important part.) This is important because it embodies the physical concept of probability, for example. That's why quantum wavefunctions need to be not only vectors, but elements of a Hilbert space. Above, I just restricted myself to the vector properties because that's what the OP seemed to want.
 One might object that we've added quantities of different units, and we're taught in physics to never do this. There's a perfectly good reason we're taught that in physics: it's basically never useful, and is typically a sign that we've done something wrong. But from a purely mathematical perspective, it's actually valid. You could add
1 meter to
5 kilograms*meters/second to get the quantity
1m + 5kg*m/s, much like you can add 1 to 5$i$ (where $i$ is the square-root of -1) to get the quantity 1 + 5$i$. I just don't know what the quantity
1m + 5kg*m/s actually means, or how it could be useful. Basically, if I got something like that in a calculation, I would know I had made a mistake because I never want anything like it. Nonetheless, it is mathematically well defined. In fact, one could argue that Clifford algebra (or geometric algebra) does essentially the same thing, by adding a scalar to a vector or bivector. If one attaches units of distance to the vector, for example, a spinor has units [dimensionless + distance$^2$]. There are other possible interpretations, but that's a valid one.
 There's also the concept of a trajectory in phase space, where the trajectory is given by a vector. Technically, this vector is not in the phase space; it's in the tangent space to the phase space. Still, that's an example of a vector space where the different directions would have different units.