If there is a potential difference between the ends of a battery, does it mean that there is always an electric field?

First of all, I am really sorry if this question is wrong. But, I thought that this would be the best place to ask this. Here is what I am thinking:

Batteries promote flow of current in a circuit due to the potential difference they create, or in other words due to the electric field that is produced due to concentration of negative charge on the negative terminal or the positive charge on the positive terminal. And, whenever any conductor is used to join positive and negative terminal, charges flow either from + to - terminal or vice versa due to the electric field being produced. Now, my question:

Due to the potential difference between the terminals, shouldn't there always be an electric field, even if we don't join the terminals using a conductor?Is this incorrect? If it is incorrect, then why is an electric field created by only joining the terminals of the battery?

A battery is basically an electron pump. Inside the battery a chemical reaction pumps electrons from the positive end to the negative end. As the electrons move they create a charge separation that opposes the pumping action, so if the battery isn't connected to anything only a limited negative charge can be built up on the cathode and a positive charge on the anode.

The amount of charge that separates is determined by the capacitance of the battery. If the battery voltage is $V$ and the capacitance is $C$ the the charge accumulated at the ends of the battery will be:

$$Q = CV$$

For a typical battery the capacitance is very, very small, so the charge built up on the ends of the battery is very, very small. However in principle the charge is not zero, and as a result a battery will be surrounded by a very, very small electrostatic field.

I have to confess I have no idea what the capacitance of a battery is, but since I've never heard of anyone measuring the field generated by a battery I would guess the field is too small to be easily measured.

Yes, there is always an electric field around a battery, even when nothing is connected to the battery.

However, if there isn't a current flowing, the electric field doesn't have much of an effect, because no energy is being expended doing anything interesting according to Watt's law, $P=VI$.

The field does exert a Lorentz force $F=qE$ on a charged object near the battery, but the force isn't very large because the field strength $E$ isn't very large given a typical battery's voltage and distance between the terminals. For example, the electric field produced by a battery is much weaker than the field involved in everyday static electricity effects, such as when you rub your feet on the carpet when the air is dry. Those everyday static electricity effects involve potential differences of thousands of volts, much larger than the 9 volts or less of a typical battery.

Another way to look at how weak the electric field is around a battery is to compare it to how strong the field would need to be in order to cause a spark to jump between the terminals. The field around a battery in reality is complicated due to the complicated geometry involved, but it's helpful in making a rough estimate to consider that in a uniform electric field, such as between two large charged parallel plates, the magnitude of the electric field is $E=V/d$, where $V$ is the voltage between the plates, and $d$ is the distance between them. The breakdown field strength of air is about $3MV/m$, which means that in order to get a spark to go between the terminals of a 9V battery, the terminals would have to be only about $3\mu m$ apart, about a thousand times closer together than they really are. Put another way, the electric field between the terminals is about a thousand times too weak to produce a spark between the terminals.