Are all methods of propulsion based on the same principle? I am struggling with coming up with a way to differentiate between propulsion on the most basic level.
Are all methods of propulsion based on throwing stuff in the opposite direction? Can the be no other method?
I started thinking about the difference in methods of propulsion when sitting on a ferry. The ferry seemed inefficient by the amount of water that it was throwing up in the air, at least compared to a car. Well, different engines have different efficiency, so that is of less interest, but is the concept of throwing water backwards essentially the same as driving a car? Is the concept the same and only the implementation or efficiency different?
All examples I can come up with ultimately boils down to pushing matter in the opposite direction:
Cars pushing the earth backwards via electromagnetic repulsion between tire and the earth, 
Rockets,
Propeller or jets on airplanes
...
Are all engines we have, based on inertia?
 A: The purpose of a propulsion system is to provide acceleration, but acceleration means the momentum of the accelerating object is changing. However momentum is a conserved quantity so if the object's momentum changes by $+p$ then the momentum of something else must change by $-p$ to keep the net change equal to zero. In this sense propulsion systems always involve throwing stuff in the opposite direction.
However the throwing can be indirect. For example in a railgun the projectile is accelerated by electromagnetic fields so it doesn't directly throw anything backwards. Nevertheless the gun/base/Earth will have their momentum changed by an amount that is equal and opposite to the momentum change of the projectile.
A: At the moment, yes.
This is due to the conservation of momentum, which does not allow for sudden movement into one direction.
However, there is the notion of exotic matter with negative mass. Such matter has not been found yet (and is no part of any current accepted physical theory). But if you could obtain some negative matter somehow and hold it together, you could build the first engine without propellant.
The setup would be a blob of negative-mass $m_-$ matter beside a blob of positive-mass $m_+$ matter. Let's calculate the forces in the system. $m_-$ acts on $m_+$ with the force
$$
F_{-+} = G \frac{m_-m_+}{r^2}
$$
which results in the acceleration
$$
a_+ = \frac{F_{-+}}{m_+} = G \frac{m_-}{r^2}
$$
$m_+$ acts on $m_-$ with the same force, but in the opposite direction (actio = reactio), which results in the acceleration
$$
a_- = \frac{F_{+-}}{m_-} = \frac{-F_{-+}}{m_-} = -\frac{m_+}{r^2}
$$
If you manage $m_- = -m_+$ in your setup, then you see, that
$$
a_- = -\frac{-m_-}{r^2} = a_+
$$
so both blobs of matter are accelerated in the same direction! This seems counter intuitive, but that is really what would happen with negative matter. It also would not violate any mechanical laws of conservation, like conservation of momentum or energy. This is because of the negative sign of mass: its momentum points in the opposite direction of movement, and its kinetic energy is negative.
There would be practical difficulties. Negative mass does repulse other negative mass, so you would have some problems holding a gravitationally significant amount of matter with negative mass together.
