You can use the formula 

$work  = force \times distance$

 to imagine it intuitively.

When the pushing force is close to the pivot, it moves through a small distance (for a given angle of rotation), the work done is low, so the kinetic energy gained by the door, related to the speed, is low.

When pushing through the same angle further from the pivot, the distance is larger, for the same angle, so the kinetic energy gained and speed of the door would be larger.