I know that the main difference between the two is that a battery can provide a constant voltage whereas a capacitor's voltage decreases as the charge stored decreases. But what about the internal structure ? A battery also has chemical reactions going inside of it while a polarized capacitor also has an electrolytic structure. So is there or is there not any difference between the internal workings of a polarized capacitor and a battery?
Batteries produce a charge difference across the terminals as a result of a chemical reaction. A chemical gets changed into another one. Even in a rechargeable battery a chemical change takes place that is reversed.
An electrolytic capacitor uses chemistry to create a thin layer with an electric field across it. The thinner layer than a "regular" capacitor allows more charge to be stored in a smaller space. But the fundamental difference is that there is no chemical change when a capacitor is charged