The change in energy of an object can be determined by the work equation, where work is the change in energy:
$$ W = F \cdot d $$
I conceptualize the transfer of energy as simply a series of small "packets" of energy being transferred at every Planck length. These small "packets" of energy add up to the total energy transferred (i.e. work). I'm not sure if this conceptualization is correct, so correct me if I am wrong.
However, it makes me wonder why the amount of energy transferred is dependent on distance and not time.
$$ m_1 = 10~kg \\ m_2 = 20~kg \\ W_1 = (10~N)\cdot(5~m) = 50~J \\ W_2 = (10~N)\cdot(5~m) = 50~J \\ W_1 = W_2 \\ t_1 \neq t_2 $$
If I apply a constant force on an object, why isn't the energy transferred at a constant rate with respect to time? The energy transfer rate varies dependent on how long it takes to cover the set distance.
In other words: why is the energy transferred consistent per unit of distance, and not per unit of time?