Intuitively, it means you imparted 100 J of energy to the object to bring it to that position.

Your confusion arises from two sources (as far as I can tell):

1. You are conflating Work with Power
2. The definition of Work you are using.

Work is always independent of time. It by definition, describes the difference in energy of a system between two states. Take your elephant and pen for example. We'll take the mass of the elephant, $m_e$, to be $1000 kg$ and the mass of the pen, $m_p$, to be $1 kg$. Now, as you stated, you exerted $1 N$ of force of each object for $100 m$. Finally, we'll assume their initial velocity was $0 m/s$. Using kinematics, we may conclude the following:

1. You applied a constant acceleration, $a = \frac{F}{m}$ of to each object. The Elephant accelerated to $\frac{1N}{1000kg} = 10^{-3} m/s^2 $ and the pen accelerated to $\frac{1N}{10kg} = 10^{-1} m/s^2 $.
2. At $100m$, each object will have accelerated to $v_f = \sqrt{v_i ^2 + 2 ad} = \sqrt{2ad}$. The Elephant accelerated to $\sqrt{2 \cdot 10^{-3} m/s^2 \cdot 10^2 m} = 0.447 m/s$. The pen accelerated to $\sqrt{2 \cdot 10^{-1} m/s^2 \cdot 10^2 m} = 4.47 m/s$.
3. It took $t = \frac{v_f}{a}$ seconds to bring each object to 100m. For the Elephant it took $\frac{0.447 m/s}{10^{-3} m/s^2} = 447s $. For the Pen it took $\frac{4.47 m/s}{10^{-1} m/s^2} = 44.7s $.

Notice: Because the elephant was 100x more massive than the pen, it took significantly more time to push it 100m, and with a constant force it was traveling at one-tenth the velocity as the pen.

Now if we calculate the calculate the difference in kinetic energy to move the pen and elephant 100m: $$T_e = \frac{1}{2} m_e v_{ef}^2 - \frac{1}{2} m_e v_{ei}^2 = 0.5 \cdot 1000kg \cdot (0.447 m/s^2)^2 - 0.5 \cdot 1000kg \cdot (0 m/s^2)^2 = 100 J$$ $$T_p = \frac{1}{2} m_e v_{ef}^2 - \frac{1}{2} m_e v_{ei}^2 = 0.5 \cdot 10kg \cdot (4.47 m/s^2)^2 - 0.5 \cdot 10kg \cdot (0 m/s^2)^2 = 100 J$$

You'll derive that the kinetic energy is identical. According to the Work-Energy theorem, the difference in kinetic energy of the system is equal to the total work done on the system.

Still confused? Take a look at the power used to move the objects:

$$P = \frac{W}{t} = \frac{Fd}{t} = Fv $$

You'll find that the power used to move the elephant was 0.447 Watts and the power used to move the pen was 4.47 Watts. You still expended the same amount of energy for either scenario; but, you expended energy at a faster rate pushing the pen, than pushing the elephant. This may seam counter-intuitive; but, remember, you exerted 1 N of force on a friction-less surface. On surfaces with friction, we must exert enough force to overcome static friction before the object will move.

Intuitively, it means you imparted 100 J of energy to the object to bring it to that position.

Your confusion arises from two sources (as far as I can tell):

1. You are conflating Work with Power
2. The definition of Work you are using.

Work is always independent of time. It by definition, describes the difference in energy of a system between two states. Take your elephant and pen for example. We'll take the mass of the elephant, $m_e$, to be $1000 kg$ and the mass of the pen, $m_p$, to be $1 kg$. Now, as you stated, you exerted $1 N$ of force of each object for $100 m$. Finally, we'll assume their initial velocity was $0 m/s$. Using kinematics, we may conclude the following:

1. You applied a constant acceleration, $a = \frac{F}{m}$ of to each object. The Elephant accelerated to $\frac{1N}{1000kg} = 10^{-3} m/s^2 $ and the pen accelerated to $\frac{1N}{1kg} = 1 m/s^2 $.
2. At $100m$, each object will have accelerated to $v_f = \sqrt{v_i ^2 + 2 ad} = \sqrt{2ad}$. The Elephant accelerated to $\sqrt{2 \cdot 10^{-3} m/s^2 \cdot 10^2 m} = 0.447 m/s$. The pen accelerated to $\sqrt{2 \cdot 1 m/s^2 \cdot 10^2 m} = 14.1 m/s$.
3. It took $t = \frac{v_f}{a}$ seconds to bring each object to 100m. For the Elephant it took $\frac{0.447 m/s}{10^{-3} m/s^2} = 447s $. For the Pen it took $\frac{14.1 m/s}{1 m/s^2} = 14.1 s $.

Notice: Because the elephant was 100x more massive than the pen, it took significantly more time to push it 100m, and with a constant force it was traveling at one-tenth the velocity as the pen.

Now if we calculate the calculate the difference in kinetic energy to move the pen and elephant 100m: $$T_e = \frac{1}{2} m_e v_{ef}^2 - \frac{1}{2} m_e v_{ei}^2 = 0.5 \cdot 1000kg \cdot (0.447 m/s^2)^2 - 0.5 \cdot 1000kg \cdot (0 m/s^2)^2 = 100 J$$ $$T_p = \frac{1}{2} m_e v_{ef}^2 - \frac{1}{2} m_e v_{ei}^2 = 0.5 \cdot 1 kg \cdot (14.1 m/s^2)^2 - 0.5 \cdot 1 kg \cdot (0 m/s^2)^2 = 100 J$$

You'll derive that the kinetic energy is identical. According to the Work-Energy theorem, the difference in kinetic energy of the system is equal to the total work done on the system.

Still confused? Take a look at the power used to move the objects:

$$P = \frac{W}{t} = \frac{Fd}{t} = Fv $$

You'll find that the power used to move the elephant was 0.224 Watts and the power used to move the pen was 7 Watts. You still expended the same amount of energy for either scenario; but, you expended energy at a faster rate pushing the pen, than pushing the elephant. This may seam counter-intuitive; but, remember, you exerted 1 N of force on a friction-less surface. On surfaces with friction, we must exert enough force to overcome static friction before the object will move.

EDIT: corrected erroneous calculations.