What is a joule? I find the definition confusing This is the definition on Wikipedia:

It is equal to the amount of work done when a force of 1 newton displaces a body through a distance of 1 metre in the direction of
the force applied.

I take that to mean, it is equal to the amount of work done when you push on an object with a force of 1 newton until the object has moved one metre.
When I imagine examples it doesn't make sense though.
Let’s say there is a small ball bearing floating in space, completely motionless relative to me. I push on it with a force of 1 newton until it moves 1 metre. OK, I've done 1 joule of work.
But now let's replace the ball bearing with a bowling ball. If I push on it with a force of 1 newton until it moves 1 metre, it will accelerate much slower. It will take much longer to move 1 m, so I'm pushing on it with a force of 1 N for longer, so I feel like I have done more work moving it compared to the ball bearing.
 A: Pushing the ball bearing with 1 N for one meter and pushing a bowling ball with 1 N for 1 meter do exactly the same amount of work: 1 joule.  As you say: it will take a much longer time for the bowling ball to move one meter. This means that at the end of the 1 meter trip the bowling ball and the ball bearing will have the same kinetic energy, but the bowling ball will have much more momentum.
Remember:
Force $\times$ distance = change in energy
Force $\times$ time = change in momentum.
Failing to make this distinction confused everyone  right up to the time of Newton, and still confuses people meeting mechanics for the first time.
A: The definition does not mean that you push the object with a force of one newton for a short amount of time and then let go and wait for it to travel one meter.
It means that you push the object and keep pushing it with a force of one newton and after the object has travelled one meter with you still pushing it you have done one joule of work. If there is no resistance to the motion (other than the objects inertia) the object will be accelerating away from you all of this time so you have to keep pushing, and hence keep doing work.
You can work out how fast your object will be going after this one meter of pushing:
$$v^2 = 2as, v = \sqrt{2as}$$
So with 1 newton, 1 kilogram and 1 meter the object will be travelling $\sqrt{2}$ meters per second. Using $KE = \frac{1}{2}mv^2$ we again get 1 joule for the kinetic energy.
If the weight is 2 kilograms the acceleration is halved. So after 1 meter the object has a velocity of $1$ meter per second and the kinetic energy will still be one joule.
An interesting observation here is that the inertial mass acts as a brake on the rate we use energy (power). With increasing mass, acceleration decreases directly and hence velocity (at a given time) decreases directly also. Therefore kinetic energy decreases with the square of time (by the velocity contribution). Conversely, it only increases directly by mass contribution, so the net effect is a reduction in the rate energy is used.
A: One way of looking at it is that a joule measures work, and works is defined as being a thing that, by definition is determined by force times distance; the ball bearing and the bowling ball have had the same work done on them, by definition of "work".
You are focused on the force times the time over which the force is applied, that is, $F\cdot t$. This is the impulse, not the work. It certainly is true that a higher impulse is applied to the bowling ball. But if you want to multiply something by time to get work, you need power. That is, work is $P \cdot t$. While you're applying power to the bowling ball longer, the power you're applying is lower. You can have lower power, even if you have the same force. Consider trying to ride a bicycle up a hill. If there are two people, one with a bicycle in a low gear, and one in a high gear, and both apply the same force, the one with a higher gear will have a higher rpm and therefore higher power.
A: 
If I push on it with a force of 1 newton until it moves 1 metre, it will accelerate much slower, it will take much longer to move 1m, so I'm pushing on it with a force of 1N for longer

If there is no friction, it's always possible to move an object by 1 m with exactly 1 J of work by applying a force of 1 N. It will indeed take much longer for heavier objects, but so what?
In the presence of friction, moving an object by 1 m with just 1 J may not be possible. If the static friction force is higher than 1 N, you will not be able to move the object at all. It doesn't mean the definition of work is flawed: you can apply a higher force if necessary, and still produce the exactly proportional amount of work.
A: Maybe imagine the ball is being pulled by a weight, or a spring, back here on Earth. The weight falls down and pulls the ball along:

It is easy to see that the weight goes down 1 meter when the ball goes along 1 meter, no matter how heavy the ball or the weight, right? But, the weight and the ball will be going a lot faster if the ball is light. A heavy ball holds back the weight, and stops it from falling as fast as it would like to.
The difference from the rocket motor, is that a rocket motor wastes energy even when it's not pushing the ball. The rocket motor uses up energy continually, every second, because it spews out hot gas. It only does one Joule of work on the ball, but it does many joules of work on the hot gas that it accelerates out the back. By contrast, the weight doesn't use up any energy, except when it actually moves.
A weight is a very convenient way to make a "pure force" here on Earth. In space, you could use a big spring anchored to a heavy space station.
A: Although you push on the heavy ball for longer, you're doing work at a slower rate, so the total change in energy is the same. The rate of change of kinetic energy for an object of constant mass is related to the rate of change of the velocity, i.e., the acceleration. We know that $F=ma$, so given a constant force, a more massive object will accelerate slower. The kinetic energy of the object equals $1/2mv^2$, so we can infer that as the object changes velocity (accelerates) more slowly, its kinetic energy changes more slowly. You can impart energy at a high rate for a short time on the small ball, or impart energy at a low rate for a long time on the big ball, and wind up with the exact same change in kinetic energy.
As a different example, imagine a potential energy case of raising a heavy object to some height. No matter whether you lift the object quickly or slowly, you're going to wind up with the exact same change in potential energy, and will therefore perform the same amount of work. In the end, it's not the time over which a force is applied that really matters for computing work, it's the distance.
A: OK, let's plug in some numbers to help visualise the situation.
Let's say the ball bearing is 1 g = 0.001 kg. Under the force of 1 N it will accelerate with 1000 m/s2. It will take approximately 0.045 s to travel 1 m so you accelerate the ball to 45 m/s velocity.
Using 1 kg for the bowling ball, it will accelerate with 1 m/s2 and it takes 1.41 s to travel 1 m. Final speed at 1 m will be 1.4 m/s.
So you are right that you have to push the bowling ball for much longer. However in the case of the ball bearing you need to exert the same 1 N force at much higher speeds, up to 45 m/s. Now, exerting the same force at a higher speed requires more power from your side, actually linearly proportional with speed. That is the intuition behind the fact that the ball bearing will require much more work than what you expect.
A: 
What is a joule?

It is unit of work, and work is a scalar quantity defined as
$$W = \int \vec{F} \cdot d\vec{s}$$
where $\vec{F}$ is force and $\vec{s}$ is displacement. Note that dot between force and displacement represents scalar product, which means that only force component parallel to displacement does work, while the perpendicular component does no work. From this definition we can conclude that unit of work is $\text{Nm}$ which is assigned a special unit called Joule: $1 \text{ J} = 1 \text{ Nm}$.

But now lets replace the ball bearing with a bowling ball. If I push on it with a force of 1 newton until it moves 1 metre, it will accelerate much slower, it will take much longer to move 1M, so I'm pushing on it with a force of 1N for longer, so I feel like I have done more work moving it compared to the ball bearing.

Everything would have made sense if you knew about the work-energy theorem
$$\Delta K = W$$
The above theorem was derived directly from the second Newton's law of motion and from the definition of work. What it says is that change of kinetic energy equals total work done on an object. Since kinetic energy is defined as $K = \frac{1}{2} m v^2$ the above equation can also be written as
$$\frac{1}{2} m (v_f^2 - v_i^2) = W$$
where $v_f$ and $v_i$ are final and initial velocities, respectively. For the same amount of work done on two objects with different mass, the speed of the heavier object will change less than speed of the lighter object. It takes more work to accelerate heavier object to the same speed difference compared to lighter object, which is kind of intuitive.

it will accelerate much slower, it will take much longer to move 1M

The common misconception is to compare work done by two forces by looking for how long they were acting. Work by definition is not about time, it is about displacement. Integral of force over time is impulse which is related to the impulse-momentum theorem
$$\vec{J} = \int \vec{F} dt \qquad \text{and} \qquad \Delta \vec{p} = \vec{J}$$
which is (slightly) different concept to work, but equally useful.
I will conclude this with a paragraph about abstract concept of energy:

"It is important to realize that in physics today, we have no knowledge of what energy is. We do not have a picture that energy comes in little blobs of a definite amount. It is not that way. However, there are formulas for calculating some numerical quantity, and when we add it all together it gives "28" -- always the same number. It is an abstract thing in that it does not tell us the mechanism or the reasons for the various formulas.
From "The Feynman Lectures on Physics", Vol 1., 4-1 What is energy?

A: I want to focus on one aspect of your question.
Intuitively you think that you have "done more work" if you push something slowly (say, a heavy object) with 1N over one meter, compared to moving something that moves easier.
Subjectively, that feeling is common. In fact, it can be quite exhausting to just hold something stationary, like in this image on Wikimedia (Kim Hansen):

No energy whatsoever is transferred to the object: it is not lifted in the gravity field, it is not accelerated, it has not moved against friction heating up the environment: Nothing of that sort. And still Atlas feels exhausted, and probably hungry!
Interestingly, this subjective feeling has an objective background: We do burn "calories" or, in SI units, Joule when we tighten our muscles, energy we get from oxidizing our food. Our body can use up energy without transferring it to another body. When we cramp our muscles without moving our limbs all the blood sugar is burned up to no effect except a muscle ache, and transformed into heat.
So if you move, say, a heavy ship very slowly with a force of (let's be more realistic) 100N over one meter, for which you needed, say, about one hour, you have somewhat accelerated it and created some heat, together making up exactly 100J; but you have also exhausted yourself, partly by just keeping yourself upright, partly by "pressing". That burned thousands of Joule on top of the actual 100J of "work done" (just standing there would burn around 100 Joule per second).
A: Taken to an extreme, your argument could be used to support the notion that we should calculate an infinite amount of work being done when we push against a static wall.
Work is an energy determined as force acting over a distance regardless of the time. The work energy translates in your example to kinetic energy in the object. Power is work per time. Consider the work, kinetic energy, object velocity, and power input for 1 N in two cases, applied for 1 s to move 1 kg by 1 m and applied for 100 s to move 100 kg by 1 m.
$w_1 = (1)(1) = 1\ \mathrm{J}$
$E_{K1} = 1\ \mathrm{J}$
$v_1^2 = 2\ (\mathrm{m/s})^2$
$P_1 = (1)(1)/(1) = 1\ \mathrm{W}$
$w_2 = (1)(1) = 1\ \mathrm{J}$
$E_{K2} = 1\ \mathrm{J}$
$v_2^2 = (2/100)\ (\mathrm{m/s})^2$
$P_2 = (1)(1)/(100) = 0.01\ \mathrm{W}$
You will do the same work for the same kinetic energy change. You will input more power to the first object, and it will have a higher velocity.
