I am aware of Maxwell Equations in Curved Spacetime. But how do these equations change if the cosmological constant is not assumed to vanish?
Adding a cosmological constant $\Lambda$ won't change Maxwell's equations (which are obtained when you vary the action with respect to the field $A_\mu$), but it will change the equations of motion in the gravity sector. See Supersymmetric, cold and lukewarm black holes in cosmological Einstein-Maxwell theory, for instance.
Also this wikipedia article.