Physicists use a narrower definition of the word "work" than normal people. We care about the amount of work done on an object and less often about the work done by something else. If work is done on an object, then that object has to change in some way, whether by changing speed, getting hotter, or changing shape.
In your example with two people pushing on a book, if the book never moves, then no work is done on it. The people will be tired after a while, and their muscles will have done work on themselves--as evidenced by the muscles getting hotter. But, this has more to do with the biology of muscles. So, energy was expended and calories from food were used up, but none of that work was done to the book.
Simpler example: imagine that same book sitting on a table. There are two forces acting on the book: gravity and the table's stiffness [1]. These forces are equal in magnitude and opposite in direction [2], so they add up to zero and the book doesn't move. If an object is not changing in some way, then there is no work being done to it.
One clue as to whether work is done is to see if the motion, or lack thereof, could continue forever without some sort of input like fuel or electricity. It takes work to accelerate an object in a straight line, since any vehicle needs fuel to do this, whether it's a car, a rocket, or a sprinter. In contrast, a book sitting on a table can remain there indefinitely, so the table and gravity are not doing work.
In a comment, you asked whether walking in a circle constitutes work. It is natural to think this, since the force causes a change in motion. Plus, being a passenger in a car making violent turns subjects one to a lot of unpleasant interactions with the seat belt and window. However, consider the planets going around the Sun. Earth has been going around the Sun for over 4 billion years and the orbital path it follows does not seem to be changing appreciably. The Sun surely burns through a lot of fuel, but it does so to release heat and light, not to affect the motion of the Earth. In fact, if the Sun were replaced with a black hole of equal mass, the Earth's orbit would be the same. This is evidence that nothing is doing work on the Earth to keep it in orbit. This is why work is defined as the movement of an object multiplied by the component of force on that object in the same direction as the motion. The gravitational force from the Sun is nearly always perpendicular to the velocity of the Earth, so no work is done [3].
[1] Technically called the "normal" force because the force is normal--i.e., at 90 degrees--to the table's surface.
[2] This is not an example of Newton's Third Law.
[3] If a planet orbits a star in a perfect circle, then no work is done at any point in the orbit. However, most orbits are elliptical, so the planet speeds up as it gets closer to the star and slows down as it gets farther away. But, since the planet is traveling at the same speed when it returns to the same position in orbit, the total work done over a complete orbit is zero.