When a potential difference is applied across a conductor, and if an electron moves from the negative terminal of the battery and reaches the positive terminal, then I want to know if the electron will remain at the positive terminal or will it again move toward the negative terminal through the battery?
Electrons that reach the positive terminal indeed remain there. The potential difference between the two terminals pushes electrons from the negative anode toward the positive cathode. When an electron reaches the cathode, it stays there to equalize the original charge imbalance between the two nodes. When electrochemical redox reaction sustaining the electron movement equilibrates, the motion will stop and the battery will "die."
As the diagram shows, the two terminals are connected by a "salt bridge." But the salt bridge is specifically designed to prevent electrons from flowing directly from the anode to the cathode. So the electrons can only flow through the circuit.
Battery is an electrolyte and positive ions can move in it rather than electrons. Similarly to the current of holes in semiconductor, directed flow of such ions complements the flow of electrons in the wire.
Here, electron-cation pairs are created at one electronde of the battary and recombine at the other (you may see that one electrode is destroyed while some material is deposed at the other by simply passing a current through the salty water).
Opposite flow of positive charges in battery/semiconductor is exactly equal to the direct current of the electrons in the conductor. You can think of positive charge moving in one direction as the current of negative charges moving in opposite direction.
Other answers just tell you that (a tiny pulse of) ion current is made first in the battery. It creates the voltage difference between anode and cathode. This causes electron current in the wire. The electrons stop at the positive electronde, thus, reducing the voltage. The battery restores the voltage by pushing more positive ions to that electrode. The flow of these ions is the current that you miss.
Samama, you've discovered a misconception being promoted by children's books. They teach that batteries are a weird sort of capacitor, with zero current in the electrolyte. They tell us that one battery plate is supplying electrons, and the other plate is a storehouse for 'used' electrons.
That's a load of complete rubbish.
In fact, the path for electric current is through the battery, through the electrolyte, then back out again. For every bit of charge that flows out of one battery terminal, equal charge must always flow through the battery, and equal charge must also flow into the other terminal. A battery is a good conductor; a short circuit. The total charge inside a battery never changes. (When a battery is "discharged," it loses chemical fuel, not charge.)
In other words, batteries are charge-pumps. They aren't sources of electric charge, any more than a water pump can create water or store water.
Do electrons flow through the electrolyte? Not necessarily, and now you've also discovered why we use "amperes" or "conventional current." Electric current isn't made of electrons. Instead, it's made of positive ions, negative ions, protons, and yes, electrons. The type of flowing charges depends on the type of conductor. To hide all this complexity, we ignore the actual flow of charges, and just look at amperes alone. Protons flow in the same direction as amperes, and electrons flow backwards. Hide the protons, hide the electrons ...and now you're thinking like an electrical engineer or a physicist. But if you want to understand batteries, you need the chemist and physics viewpoint.
In many batteries, electrons do flow in the metal parts, but only protons flow in the electrolyte! The proton-drift is a perfectly valid electric current, and it can even be an enormous current (for example in car batteries, hundreds of amperes of proton-flow are taking place in there.)
Here's one simplified view: during an electric current in a circuit, the amperes inside the battery-electrolyte is actually made of protons flowing backwards. At the negative electrode, violent chemical reactions tear water apart into individual atoms. Oxygen gas is released, but the hydrogen atoms each have an electron torn off, and these electrons get pushed into the metal surface, charging it negative. The hydrogen atoms are missing their electron, so they've become +H positive hydrogen ions. The electrolyte is a good conductor, and all the +H hydrogen ions are attracted to the other distant plate. They flow through the electrolyte (many amperes.) Then, at the other plate, the +H hydrogen ions combine with electrons from the metal surface. Some energy is used up, and neutral hydrogen gas is produced. And in total, if there is an ampere in the metal wires, there must be an ampere of ion-flow in the electrolyte.
The above is a description of fuel cells, and also of lead-acid batteries. In these, the electric current in the electrolyte is a proton flow! (The +H positive ions have another name: PROTON.)
If instead you want to avoid this backwards charge-drift, with all the canceling charges and water-splitting, instead just look at Alkaline batteries. In these, the mobile ions are -OH ions, not positive protons. We can view them as electrons which are hitching a ride on a piece of a water molecule. In that case the entire electric circuit is a flow of negative charge, including the current in the water between the plates. (But that's not a universal rule, and other batteries have proton-flows instead. Or, if you make a simple battery using salt-water, then you have two opposite flows in the electrolyte, the positive +Na ions going past the -Cl ions, both at the same time.)
But why should charges flow at all?
In other words, why do batteries have a voltage? Ah, that explanation comes from chemistry. It happens because the reaction at the negative plate is violent, energetic, exothermic, and spontaneous. Stick some metal into water and the water attacks! This reaction is the battery's energy-source.
Batteries are something like fire: they're "burning" some fuel-chemicals, pumping an electric current which can drive an external device, and leaving behind some waste products. Each battery is like a tiny power plant, complete with flames and a boiler and some rotating magnets with coils! Well, not exactly. Instead, batteries are more like tiny little VandeGraaff machines, they're mechanical charge-pumps driven by spontaneous exothermic chemistry. At the negative terminal, energy-producing reactions are forcing electric charges to move across the metal/water contact region. (Instead of being exothermic, most of the energy from the chemical reactions is ending up as electromagnetism, as energy stored in e-fields.)
It's almost magical, because if the external circuit is broken, the violent spontaneous reactions stop!!! It's like having a power plant where, if you break the electrical connections, the whole thing halts instantly, and the fuel stops being burned. Even more magical: if we force the charges to flow backwards through the power-plant, the chemical waste products get turned back into fuel! (For example, in a fuel-cell, the water gets un-burned, forming hydrogen and oxygen.)
If you want to get into battery-science, some keywords to follow are: "Helmholtz Double Layer." That's where all the magic happens, in a molecule-thin layer where the water touches the negative battery electrode. Also search "Grotthuss Mechanism," that's the little-discussed phenomenon where protons flow through water as an electric current. (Heh, every grade-school textbook on earth should be teach us about the Grotthuss effect! Instead, they pretend that salt-water inside batteries is an insulator.)
In an ideal electrochemical cell case, it should remain at the positive terminal. The salt bridge should only allow for the ions to flow.
However, practical batteries have other physical phenomenon that restrict the flow of current, and hence introduce this 'internal resistance'. There could be a wide variety of reasons depending on the type of cell used. I believe one of the most common reason is polarisation. But that too, shouldn't cause the electrons to flow through the cell.
To quote a good explanation from University of Illinois Physics Dept Q&A page:
You might wonder why the electrons don't just flow back through the battery, until the charge changes enough to make the voltage zero. The reason is that an electron can't move from one side to the other inside the battery without a chemical reaction occurring. In other words, inside the battery plain electrons can’t travel around because it takes too much energy to put a plain electron in solution. Electrons can only travel inside the battery via charged chemicals, ions, which can dissolve off the electrodes. The chemical reaction is what pushes the electrons inside toward the negative end, because the electrodes at the two ends are made of different materials, which have different chemical stabilities. So overall, electrons flow AROUND the circuit, toward the negative end inside the battery, pushed by the chemical reaction, and toward the positive end in the outside circuit, pushed by the electrical voltage.
Electrical current can flow in the other way in the battery too, if the battery is hooked up to something with a bigger voltage difference (a battery charger, for example).
EDIT: As to why there is current flow inside the battery: Electrons are not necessary for current to flow. The flow of ions does happen inside the battery.