I wonder if what you're missing is that potential difference (voltage) is defined in terms of a positive test charge: The increase in potential in going from A to B is the work needed (because of the presence of an electric field) per unit test charge to move a positive test charge from A to B.
And, as you well know, an electron has a negative charge, so it gains electrical potential energy as it goes from the + to the – terminal, though the – terminal is at a lower potential, because a positive test charge would lose electrical potential energy in going from the + terminal to the –.
The drop in potential is dependent on the more or less arbitrary convention of a positive test charge, whereas the electron's gain in electrical potential energy can is not a matter of convention: the chemical mechanism of the battery has to do work pushing the electron from the + plate to the — plate against the force due to the electric field. There is therefore no contradiction in the Physics.