Suppose a system has two parts, $A$ & $B$. They were initially at different temperatures and hadn't achieved mechanical equilibrium. After attainting thermodynamic equilibrium, do they continue to exchange heat energy? ie. after attaining thermal equilibrium, ie. Is the heat that $A$ gains from $B$ the same as the heat gained by $B$ from $A$ so that the temperature remains constant ? Meaning to say, is thermodynamic equilibrium dynamic like chemical equilibrium? Or is it static? What is the reason for any of the two?
It is a dynamic equilibrium. To see this notice that the Boltzmann distribution law is statistical in nature, for instance, the molecules that have energies between $E_1$ and $E_1+\Delta E_1$ at time $t_1$ are not necessarily the same than the ones in between those same energies at time $t_2$. Particles collide (and thus exchange kinetic energy) all the time, and while the distribution stays the same, the particles that contribute to different parts of the distribution keep changing.
Update: It is dynamic, like chemical equilibrium, because the molecules do not all have the same kinetic energy. If, at equilibrium, you randomly pick a molecule, there some probability (from the Boltzmann distribution) that it will have its energy between $E_1+\Delta E_1$. If after some time $\Delta t$ you pick those same molecules again, there is again the same probability to find those molecules with an energy between $E_1$ and $E_1+\Delta E_1$ is the same, but those same molecules will likely have a different energy than the one you measured earlier. this is because they molecules collide all the time and exchange energy with each other. From a macroscopic point of view the two states, at times $t_1$ and $t_1+\Delta t$, are the same; from a microscopic point of view, the two states are different. Thus it is a macroscopic equilibrium not a microscopic one. Same when have ice and water at equilibrium. The molecules that form part of the ice or part of the water keep changing all the time. At equilibrium, you keep the same amounts of water and ice, but the molecules that make up the ice and the water keep being exchanged between the ice and the water.
Equilibrium is dynamic microscopically but macroscopically it is static because net flow is zero.