I have a matrix of complex numbers for the electric field inside a medium. Since I want to draw the quiver plot of these elements, it will be completely different if I only use the absolute part. Then if I take only real or imaginary parts, the configuration of the field will change?
The field configuration is provided by your complex data. The way you plot them does not change di field configuration but it may provide a more or less faithful visual representation of your data.
I assume that you really need the information contained in the complex field (are you sure that the physical quantity you are interested is not separately the real and imaginary part of it?).
Quiver plots of a scalar complex field use two-dimensional arrows to encode the value of real and imaginary part of the field at each point as two orthogonal components of that arrow. In the case of a complex vector field there is the problem representing two or three vector component, each with real and imaginary values. Here, I do not know if there is an established tradition. I would use a color code, visualizing each cartesian component as a 2D arrow of a different color.
As a second idea, you could do a absolute value quiver plot on top of a phase angle quiver plot with $\arg E_x$ on $x$ axis and $\arg E_y$ on $y$ axis, just like the real part in blue, imaginary part in red plot in the above suggestions. But I guess this plot will be more informative in that phase and magnitude are more physical than real or imaginary parts.
TL;DR: Just do as David says, but with quiver 1: [abs(Ex),abs(Ey)] in blue, quiver 1: [angle(Ex),angle(Ey)] in red.