It's true that traversable wormholes would theoretically allow for travel into the past according to general relativity (though this may be ruled out by quantum effects in whatever theory of quantum gravity eventually replaces general relativity, see the chronology protection conjecture). But the only way this would happen is if one mouth of the wormhole experiences time dilation relative to the other, either because it's taken on a journey at relativistic speed relative to the other mouth, or because it's moved closer to a source of gravity where it experiences gravitational time dilation. If the two mouths have clocks moving alongside them which were initially synchronized, then if some time dilation has accumulated, when the two mouths are brought back together an outside observer looking at them side-by-side will see the clocks showing different times. But, as explained by physicist Kip Thorne (who discovered traversable wormholes as a theoretical possibility in general relativity) in his book Black Holes and Time Warps, time threads differently through the wormhole, in such a way that if you look through one mouth at the clock which is alongside the other mouth, and compare it with the clock alongside the mouth you're looking through, then the clocks will still be synchronized (assuming the time for light to travel through the wormhole from the clock to your eyes is negligible). This means for example that if a side-by-side comparison by an outside observer (one who was not looking through either wormhole) showed mouth #1's clock to read 2015 and mouth #2's to read 2010, then if you jumped through mouth #2 you would exit mouth #1 when its clock also read 2010, not 2015. In some cases this would make it so that when you jump through a wormhole, you could end up in a region of spacetime where it would be possible to send a signal to your own younger self, which would be received at an earlier time then it was sent according to a clock you carried along with you (i.e. in terms of your own proper time).
But you don't mention anything about time dilation--if one mouth of the wormhole is moved slowly from A to B, so that no time difference accumulates, then there will be no time travel in your scenario. One thing to keep in mind is that in relativity, simultaneity is not generally defined in terms of when you see the light from events--for example, in an inertial frame in special relativity, if I see the light from an explosion 10 light-years away in the year 2020 (in the space and time coordinates of that frame), I subtract out the light travel time and say the event actually took place in 2010 in my frame. And wormhole spacetimes can be asymptotically flat, meaning to a good approximation you can treat them as localized distortions of spacetime moving around in an otherwise flat spacetime, so you can still set up something very close to an inertial frame in the region outside the wormholes. In this case, we might say that in a frame where A and B are at rest, the event of the clocks at A and B each reading 12:00:00 AM on Jan 1. 2020 are simultaneous. Then if I jump into mouth B at that moment, and exit from mouth A 4 seconds later according to the time coordinates of this frame, we can predict that I will see the clock at A reading 12:00:04 AM, Jan. 1 2020. If I immediately look back through the wormhole I will see the clock at B reading the same time of 12:00:04 AM, Jan. 1 2020, but if I immediately look at B through normal space using a telescope, due to the 10-light-year distance I will see the clock at B reading 12:00:04 AM, Jan 1. 2010. Assuming I had been living near B until I jumped through that mouth in 2020, looking through my telescope will also allow me to see myself when I was ten years younger.
But the main thing to realize is that there's no way I can actually send a signal to reach my younger self in the past in this scenario, as was possible in the other scenario where time dilation had created a time difference between clocks next to each mouth. If I send it through the wormhole, then (assuming a negligible travel time for a light signal) it exits mouth B 12:00:04 AM Jan. 1 2020, whereas I had jumped through mouth B 4 seconds earlier at 12:00:00 AM Jan. 1 2020. Meanwhile, if I send a light signal through normal space at 12:00:04 AM Jan 1 2020, in my frame it takes 10 years to get there, and another 10 years for the light from the event of someone receiving it at B to get back to me. This means I won't see anyone at B receive it (at least not if I am looking through normal space) until my clock reads 12:00:04 AM Jan 1 2040, and I will see the clock at B reading 12:00:04 AM Jan. 1 2030 when someone next to it receives my signal. So regardless of whether I send a signal through normal space or back through the wormhole, in this scenario the signal will arrive at B when the clock there shows a later time than the moment I jumped through the wormhole to travel from B to A.