First of all, I am really sorry if this question is wrong. But, I thought that this would be the best place to ask this. Here is what I am thinking:

Batteries promote flow of current in a circuit due to the potential difference they create, or in other words due to the electric field that is produced due to concentration of negative charge on the negative terminal or the positive charge on the positive terminal. And, whenever any conductor is used to join positive and negative terminal, charges flow either from + to - terminal or vice versa due to the electric field being produced. Now, my question:

Due to the potential difference between the terminals, shouldn't there always be an electric field, even if we don't join the terminals using a conductor?Is this incorrect? If it is incorrect, then why is an electric field created by only joining the terminals of the battery?