A ball with constant velocity has no force acting on it so it can't exert force on wall either and when that happens the wall shouldn't exert any force on it (action-reaction; wall will only exert a force on ball when ball exerts a force on it and if that is not the case then almost anything can exert force on any other thing whether or not it is in contact) and the wall is also at rest (it might accelerate backwards when it experiences force by ball but ball is exerting no force)
You seem to be assuming that the 'forces' are conserved. i.e. that no forces prior to collision implies no forces during collision. That isn't so; e.g., energy is conserved, but 'forces' are not.
When the ball is far away from the wall, the ball and the wall are both neutral and the structure of their internal charges (electrons and protons) is irrelevant. There are no net electromagnetic forces between the ball and wall and no acceleration.
When the ball gets very close to the wall, the arrangement of electrons and protons becomes relevant, resulting in repulsion between the ball and the wall. It is this electromagnetic repulsion that results in a force upon the wall and ball and accelerates the ball back and away from the wall.
With e.g., a tennis ball, a substantial fraction of the initial kinetic energy is converted first to potential energy as the forces deform the ball, then back to kinetic energy. A ball like that is said to have a high coefficient of restitution. There may, however, be energy losses to e.g. heat, sound or spin, or permanent deformation/destruction of the ball.
A ball with constant velocity has no force acting on it [...]
[...] so it can't exert force on wall either
Remember that a change in direction also counts as a velocity change. And any change in velocity - be it direction or magnitude - is called an acceleration. And force is defined as something which causes acceleration. That's Newton's 2nd law: $\sum F=ma$.
When the ball hits the wall, the wall pushes on it to stop it. The wall exerts a normal force, which is a force a surface exerts when necessary to avoid breaking or deforming. If it didn't exert any force to stop the ball, the ball would pass right through the wall.
According to Newton's 3rd law, the ball replies with an equal but opposite reaction force, which pushes on the wall. The wall (or the Earth) then does move a tiny bit, as you mention, but negligibly little.