How can a metal and an insulator both have high dielectric constants, yet one is conducting while the other is insulating? I don't get it: insulators are used as dielectrics. The higher the dielectric constant, the better the insulator is. 
However, the dielectric constant of metals is considered to be infinite. 
Doesn't that make metals perfect insulators? 
 A: Hi. 
I'm first going to explain what the dielectric constant is, since I think you may be a bit confused. Then I'll answer the question as to why we say metals have infinite dielectric constants.
I hope you're familiar with parallel plate capacitors and capacitance, since I think that's the easiest way to understand this. At least that's where I first learned about dielectric constants. If you're not familiar with capacitance, I suggest you look at that before, then come back - my answer relies heavily on it.
Hope it helps!

First of all, what is the "dielectric constant?" 

It's the number that the capacitance of a parallel plate capacitor
  gets scaled by when you put in some material between the two plates.

But that definition is super-boring and gives no physical intuition. 
Why does the capacitance of the capacitor change in the first place when we put in some material between its plates? 
There's got to be a deeper answer. 
What the dielectric constant is truly measuring is how polarizable atoms in a material are under the influence of an electric field. 
If that sentence made absolutely no sense, don't worry! I'll explain.
When you put a material in an electric field (ie: between the two oppositely charged plates of a parallel plate capacitor), electrons inside the material will want to move against the electric field (towards the positively charged plate).
However, for materials like rubber or glass, the electrons are bound way too powerfully to their respective atoms to move far without being pulled back into place. They'll only shift a little bit. This shifting will create a tiny bit of an electric field that opposes the electric field between the parallel capacitor plates, and thus reduces the overall electric field between the plate by some constant. Let's call this constant, the amount the electric field between the plates gets reduced by, $p$.
THE DIELECTRIC CONSTANT CONSTANT IS THE RECIPROCAL OF $p$. - Just keep reading to see why. 
Here, a picture of polarization:

Now, how does this connect with "...the number by which the capacitance gets scaled by..."?
Say we've got some parallel plate capacitor with a charge of $Q$ on its plates causing a voltage of $V$ between them. 
We now put in a dielectric between the two plates. When we do this, the magnitude of the electric field between the parallel plate capacitors gets scaled by a factor $p$ (note that $p<1$) due to the opposing electric field caused polarization of the atoms within the dielectric. 
The voltage between the two plates is equal to the electric field between the two plates multiplied by the distance between them. The distance didn't change. Therefore, the voltage between the two plates will have gotten scaled by that same constant: $p$.
However, the charge on the plates won't have changed. The same charge $Q$ on the plates that previously caused a voltage $V$ now cause a lesser voltage: $pV$. 
Thus, the capacitance of the capacitor will have gone from $\frac{Q}{V}$ to $\frac{1}{p} \frac{Q}{V}$. In other words, it will have gotten divided by $p$, or multiplied by $\frac{1}{p}$. 
$\frac{1}{p}$ is the dielectric constant of the material we stuck between the plates. We usually call this constant $k$.
If your capacitor is hooked up to a battery, instead of viewing the dielectric constant as the reciprocal of the factor the voltage between the plates gets scaled by when we stick in the material between them, you instead have to view it as the factor that the charge on the plates will increase by. Take a moment to think about it! 

Now for the metals and their infinite capacitance:
When we put in a block of metal between two parallel plates of opposite charge (generating a nearly constant electric field between them), free charges (electrons) inside the metal will move under the influence of the electric field. 
In nonconductive materials, there weren't any free charges, which is why we got polarization. 
Charges will move to the surface of the metal block – negative charges to the side nearer to the positively charged plate, and positive charges to the side nearer to the negatively charged plate (in other words, some electrons will leave one side of the metal block and some other electrons will spread out over the other side of the metal block).
This will continue to happen until the charge density on the surface of each side of the metal block is equal to the charge density on the capacitor plate that side is closest to, but opposite in sign. When this charge density is achieved, the electric field inside the metal block becomes zero, and the electrons have no reason to keep on moving.
Here, you've probably seen a picture like this before, showing field lines ending on one side of a metal and coming out of the other:

The reason the field lines end at the surface of metals in electric fields is exactly because of what I've said above - charges within metals are free to move, and they will do so until there is no net field within the metal.
The metal block will have reduced the voltage between the plates. How? 
The magnitude of the electric field in the spaces between the metal block and the plates won’t have changed, but inside the metal block there’s no electric field.
Remember the definition of voltage? Its the product of the electric field and the distance. 
Well, now it's just like we had reduced the distance between the plates, since part of the space between the plates the space filled with the metal won't have an electric field inside it. 
And so, just like with the polarizable dielectric material that decreased the voltage by decreasing the electric field between the plates, the metal decreased the voltage by not only decreasing the electric field, but bringing it down to zero in a section of space between the two plates.
Say the capacitor plates original had a voltage $V_0$, and a separation $d$ between them, and we insert in between the two plates a metal block of thickness $t$, then the new voltage (assuming we don’t allow the charge density to change on the capacitor plates) will be $V_1=V_0 (1-\frac{t}{d})$. 
(I’ll leave it as an exercise for you to convince yourself of this…it shouldn’t be too hard.)
If $t$ is really, really close to $d$, the voltage between the plates approaches zero (and if $t=d$, then the metal block is touching both sheets and the capacitor just turned into a section of wire with a super thick cross-sectional area - just ignore that scenario). 
With other nonconductive (insulating) materials, we’d be able to fill in the space between the plates, and sure – the polarization of atoms within the materials would decrease the magnitude of the electric field within the plates by some constant and thus decrease the voltage difference between the plates (causing an increase in the capacitance of the capacitor, since now with the decreased electric field the same amount of charge on the plates causes less of a voltage...blablabla, like I explained up top).
But since metals have free electrons, if we put a block of metal in between two oppositely charged plates, it won’t just partly block the electric field – it’ll block it entirely in the space inside of the metal! 
That’s why we say metals have infinite dielectric constants - they don't just polarize and block the electric field a bit - they do so entirely!
Hope that helped!
