tl;dr Batteries do not create electric fields to move charges. They move charges, which creates electric fields.
a battery [...] gives out some electric field that moves through the circuit and gives a force on electrons in conductor to produce current.
This description is, if not completely wrong, at least misleading. A battery is not a source of electric field, it is a source of electric potential. Imagine a battery with terminals in the shape of a pair of parallel conductive plates with an air gap in between (this is a capacitor): as you move the plates toward each other, the field strength (between the plates) increases, and as you pull them apart, the field strength decreases. There is no upper limit to how strong the field can be (well, until it reaches the breakdown voltage and begins to arc), and no lower limit either -- the strength of the field is not determined by the battery. So the battery itself does not directly create an electric field between its terminals.
Moreover, electric field does not "move through" a circuit; charges do. In a simple DC setup like a battery driving current through a resistor, all electric fields are stable over time -- the charges move through the circuit, but the field itself does not. Saying that electric field moves through a circuit is a bit like saying that gravity moves through a rollercoaster. Moving electric fields do come into play with AC circuits and devices with moving parts, such as electric motors; but even a non-moving electric field causes charges to move through a conductor (this is, after all, essentially the definition of "conductor").
So how does current work? Batteries are a source of electric potential, which is measured in volts. Potential is a kind of pressure and in a typical battery this pressure is caused by chemical reactions inside the battery pumping electrons from one terminal (+) to the opposite one (-). The potential difference between the terminals does create an electric field. In the experiment above where the terminals of the battery are parallel flat plates separated by a distance $d$, you could calculate the field strength between them as simply $E = {V \over d}$. But the field is just a way of observing the potential difference between the terminals: it's not the reason the charges are moving in the first place (which is, again, the chemical reactions happening inside the battery).
Pressure is relative, and electric potential is no different. To be strictly accurate, what a battery provides is an electric potential difference ("voltage") between its two terminals. This is basically a measurement of how strong an electron-pump the battery has inside it. This is why connecting two identical batteries in parallel does not add their voltages: because they both provide the same potential difference between the (+) and (-) terminals, there is essentially no difference between two batteries connected in parallel and two isolated, disconnected batteries. The electric potential between the (+) terminals and (-) terminals is the same, and both batteries "agree" on it, so there is no reason for current to flow between them. (Connecting two mismatched batteries in parallel will cause current to flow, in a direction determined by which battery has higher voltage.)
What connecting two batteries in parallel does do is change how the system behaves when under load. If you connect a load, say a 1kΩ resistor, across the terminals of a single 1.5-volt battery, the current through the resistor will be 1.5 mA, all of which is supplied by the same battery. If you connect the same load across the terminals of two 1.5-volt batteries connected in parallel, the current through the resistor will still be 1.5 mA, but now each battery only has to supply 0.75 mA of current. This means that each individual battery is under less load than before, because the electrochemical pumps inside it only have to move half as many electrons to maintain the same voltage. These batteries may last longer and behave better under a variety of loading conditions. (This assumes ideal conditions and perfectly matched batteries. In real-world scenarios, batteries are never perfectly matched, and so you may need a load-balancing circuit to protect the batteries from each other.)