Premise is correct. The electric field would be the same. But lets see how this works out through an example.
Let's take two parallel plates, separated by vacuum, and connect them to a battery of potential V. The electric field between the plates will be V/d. Let's now introduce a material that has dielectric constant k.
As soon as you insert a dielectric slab between the plates, the slab will get polarised and oppose the external electric field. As a result, the net electric field inside the slab will become E/k. This will cause the potential difference across the plates to become V/k.
The plates and the battery are acting like two oppositely connected cells. Since the potential drop across the plates (V/k) has become less than that of the battery (V), current will flow until the potential difference between the plates becomes V.
As a result, Q(k-1) charge will be drained out of the battery, so that the final potential difference between the plates becomes V again.
In effect, the net electric field and V across the dielectric is the same as that across the vacuum, but the charge stored in the capacitor has become kQ now. So, the capacitance (Q/V) has increased with the introduction of a dielectric.