First of all, I appreciate your doubt. It is conceptual and involves critical thinking regarding the topic.
In order to understand work, you need to understand the 2 physical quantities namely - Force and Displacement .
Note that using only one single word Displacement in order to define work is not appropriate. (Same is for Force ) . As, Displacement depends on the direction of Force . Now, again, it will not be appropriate to just write Force in order to define Work/Displacement. I will better write it as : As, Displacement depends on the direction of Force applied on the point of application.
Whenever a body is moving, there are many forces acting upon it. While you are walking, the body is applying force on you, the ground is applying force on you, the air is resisting you from moving etc. So, basically it is much more complicated than we think.
Coming to the main point, displacement! Let us just take a point object in order to understand the situation well. Now, before I go on, I will first define what is point-object.
The concept of a point object can be understood here : Point Particle
So, as you're clear with the basic concepts, I can go on. Each object is made up of millions of particles...Uncountable! Now, when you push an object, suppose a car, then the car is moving but the force has acted upon on a specific point. That point may be referred to point of application of the force. When the car moves (if the other opposing forces become lesser than the force applied by you) , then it surely displaces itself. Thus, it does some Work. While, you have also done the work to push the object, but we usually calculate the work done by the object experiencing the force.
The displacement here, will be based on the point particle though many simply refer it to the complete object. It is not wrong either, but just a little mis-conceptional to refer the displacement to a whole object. You must keep this in mind that basically, the force(external) is acting upon the point particle which later on affects the nature of the whole object, that's why the object displaces itself.
This can be explained in 1-2 para also, but, what I'm trying to explain is critically-thinking a topic. Whenever you try to follow a question or topic, first think about it. Try to compare the conditions with nature, notice the similarities and differences. This will make your concepts much stronger!