IMO you're right: the problem's wording is bad. It's meaningless to say "the engine exerts a force of 40000 N". The engine interacts both with the first wagon and with the rail, exerting forces on both and these forces are different. How can we know what the book's author had in mind? If the former, your answer is right, if the latter, the book's one is right. Of course in the latter case (force exerted on rail) it's not that force which accelerates the train, but the rail's reaction.
As a general rule we shouldn't say "A exerts a force" - much better "A exerts a force on B". Better yet "A exerts a force on point P of B". The point where a force acts can make a great difference.
Edit
Many things have been said but IMO it's useful to be still clearer. After further reflections I think it's not wording that's bad, but the thinking behind.
As already stated e.g. by @Farcher in mechanics problems the first thing to be done is to make clear which the (mechanical) system is. In our case, it's wagons or wagons + engine? Note that it wouldn't make
difference as to acceleration, which is the same for all parts of the train, but isn't by no means the same if forces are concerned. Consider the two choices:
a) The system consists of wagons alone. Then it makes sense to say that engine (which doesn't belong to the system) applies a given force $F$ to the system and compute acceleration as $a=F/m$, where
$m=10000\,\rm kg$. Given the book's answer, this isn't the author's choice.
b) The system consists of the whole train, engine included. Then there are two alternatives as to the given force $F$ (here I'm reiterating)
b1) $F$ is a force engine applies to wagons. Then it's an internal force and as such it cannot accelerate the system. Its acceleration is only due to the net external force (in computing the resultant internal forces cancel thanks to third Newton's law). Then the
solution is nonsense.
b2) $F$ is a force engine applies to external world. In that case it will never accelerate the system or a part of it. It could accelerate the rail, were it not firmly fixed to ground. Not to mention that a force engine applies to rail exists, but is in the wrong direction.
So not even choice b) is acceptable. Neither as it's used in gandalf61's answer.
I'd like to add another comment. Unfortunately "engine" in English may mean two different things: either the motor proper (electric, diesel,
or else) which ultimately is the cause of motion - or the car carrying it. Of course the datum of mass suggests the latter interpretation. But my suspicion is that the author - maybe without realizing - thought
of the former.
I'm led to say so because it's a very common way of thinking about cars and other motorized vehicles. If you ask someone "which force causes a car move?" I bet you'll get as an answer "the motor's force". Nobody thinks that in strict mechanical meaning the motor can only produce internal forces and as such will never move a car by a cm. That the only external force which can put a car in motion is the road's friction on the wheels.