Is it wrong to just say that things continue at constant velocity unless acted upon, as velocity is only relative? Like when saying that people mistakenly thought things automatically slowed down after being thrown etc (accelerated), because that's just due to all the air or surface friction or gravity. Do you have to specify the reference frame? And no reference frame is more true?
 A: Not wrong. Newton laws was formulated for absolute space and inertial reference frames. All inertial reference frames are equal, so as long as you will analyze law of inertia from any inertial reference frame - it will hold, because it's just a matter of Galilean transformation $u'=u+v$. But, of course if you are moving for example in accelerating rocket, then law of inertia for some object thrown out of rocket window will not hold anymore, and you will need to account for your rocket acceleration- adding pseudo force to an object.
If you like, for the sake of peace, you can look around for an absolute space (absolute reference frame) postulated as axiom in Newton theory,- it can be for example a cosmic microwave background, or take a reference most distant galaxy or something similar.
A: If you are riding in a car and look out of the window, are the trees moving (or is the car moving and the trees are still)?
If you accelerate in that car while looking out of the window, do the trees accelerate (or are they still not moving and the car is accelerating)?
If you said that the car is accelerating and the trees are not, you are using a(n approximately) inertial reference frame where things really do continue at a constant velocity unless acted upon.
If you said that the trees are accelerating, then you are using a non-inertial reference frame where trees are able to accelerate without any force being applied to them.
The default for most people is to use an inertial reference frame, so it's easy to gloss over the fact that that's actually an assumption.
A: 
Do you have to specify the reference frame?

Yes, it is necessary. Velocity indeed has a meaning only as a relative quantity. However, in any non-inertial system, free bodies, i.e., bodies with no action by other bodies, do not move at constant velocity.
Things can be seen in the following way: the acceleration of a body, in general, depends on the position and velocities of other bodies and on the reference frame. However, there is a particular class of reference frames, the inertial frames, characterized by the property that a test particle moves with constant velocity in the absence of other bodies or far enough from them. In a non-inertial system, this is not true anymore.
Notice that the inertial reference systems are not more true than others. They are simpler because accelerations are only due to the action of other bodies.
A: In Newtonian terminology we say that objects will continue at constant speed in a fixed direction if they are not subjected to any force- that is what Newton's first law tells us. (You need to bear in mind, however, that Newton's laws assume that gravity is a force. In General Relativity, gravity is not considered a force but an effect due to the curvature of spacetime, so in the context of GR we would phrase things differently.)
To quantify the speed and direction, you need to specify some frame of reference.
As far as we know, there is no 'absolute' frame of reference.
Physicists usually pick frames of reference that simplify the particular calculations they need to perform. For example, when studying the relative movement of two objects in Special Relativity we usually pick a frame in which one object is stationary and the x-axis is aligned along the direction of the relative motion of the two objects. We could pick any other reference frame, but that would make the calculations more complicated.
A: 
Do you have to specify the reference frame? And no reference frame is more true?

Velocity is not an absolute variable, it is frame dependent. The second question I do not understand, please clarify.

Is it wrong to just say that things continue at constant velocity unless acted upon, as velocity is only relative?

It is not wrong - this is exactly what the first Newton's law of motion states: if the net force on an object is zero, the object will be in equilibrium. The equilibrium means that object is moving at constant velocity, which can be zero but does not have to!
Let’s say that the velocity measured from the ground (Earth) is zero. In some other reference frame it might not be zero, but it is constant if this reference frame is also inertial, i.e. if it does not accelerate. The first Newton’s law is valid in both frames.
Please note that the ground (Earth) is not true inertial reference frame, but it is considered as one within some margin of error. The Earth rotates around its axis and orbits around the Sun, which gives it (very small) acceleration.
A: This will repeat others' ideas, but introduce some hopefully helpful terminology. The Aristotelian idea that with no force velocity isn't conserved shouldn't be conflated with the Newtonian idea it will be conserved, but qua Galileo not invariant. Whereas just about any reference frame change will change quantities that aren't invariant, without a force the velocity is conserved as long as we only work with, and therefore only change between, inertial reference frames.
