Think of a water pipe system.
- The resistor is like a filter. It allows some water through (from any direction) but slows it down.
- The battery is like a pump. It literally forces water from one side to the other, and allows no water to flow the opposite way.
A battery is a fairly complex electrochemical device, but it does indeed act as a one-way charge mover, which via high chemical potentials separate electrons from atoms at one terminal and reunites them at the other. This causes a constant surplus of negative charge at one terminal and a lack at the other, and this charge difference corresponds to an electrical potential energy difference, measured as voltage. Think of voltage as water pressure difference.
Those surplus electrons want to move, and the only available path they have is through the circuit (and thus they constitute a current - think of current as the water flow) because they aren't able to move through the liquid interior of the battery. This liquid (or maybe in the future non-liquid) electrolyte, as it is called, is an ion conductor, but not an electron conductor.
Sure, rechargeable batteries (accumulators) are not "one-way streets", so the analogy is not perfect and depends a lot on the exact conditions. But the one-way pump idea works fairly well in most cases.