What makes an insulator good for polarizing? Water in an electric field become polarized easily, since the natural dipoles twist and turn to align with the field.
In non-polar insulating materials, a redistribution of charge can happen in the material when placed in an electric field, which gives the material induced dipoles. And so polarization happens anyway since these induced dipoles in molecules and particles can twist and turn to align with the field.
I am curious as to what in a non-polar material makes it a good material to be polarized? And what is the difference in the opposite case, when a material is very difficult to polarize?
 A: The most important factor for getting a large polarization in a solid is how close the material is to a distorted crystal structure which breaks inversion symmetry (e.g., a ferroelectric or piezoelectric instability). The closer a material is to being a ferroelectric etc., the larger the polarization because it is easier for the external electric field to distort bonds.
To understand this, think back to water where you can easily align all the individual H$_2$O dipole moments since each molecule is free to move. In solids, you can try to do the same and polarize bonds by applying an electric field. However, most materials are structurally very stable, so you don't see much change in bonding and thus a relatively overall polarization. When a material is close to a inversion symmetry-breaking phase transition, they have bonds that only need a slight electric field to tip them over the edge and polarize them. The closer they are to the phase transition temperature, the larger polarization you can get.
Ceramic oxides (Lead, Barium, Strontium Titanate, see image below) are good examples of solids where the bonds are easily distorted to create oriented dipole moments. In this family of materials you can get very large dielectric constants ($\epsilon_r \gg 5000$) when you are closed to the Ferroelectric phase transition.

A: It has to do with the structure of the material. Polarization, or polarization density, has to do with the size of dipole moments you can have in your material. The manner in which dipole moments are created depends on the material. 
In the example of water, each water molecule has it's own dipole moment. Macroscopically, they may form a net polarization if they are all aligned, and not so much if they are mostly random.
In non-polar media, you can still have a polarization. For example, Nobel gasses are weakly polarizable, even though they are monoatomic and have spherical electron distributions. In an electric field, the spherically distributed electrons begin to favour one side, like in a Van Der Waals interaction.
But, what really matters is the size of the polarization. A large dipole moment in a covalent bond (such as in water) can occur because of differences in electronegativity. Big differences in  electronegativity, or a better arrangement of the dipoles in the atom can lead to a greater dipole moment. In a non-polar molecule, the larger the atom, the easier it is to polarize. 
In solids, there is a slightly different story. Ice is fairly rigid in the orientation of it's constituent molecules. However, polymers like rubber has a much smaller dipole moment, but it's molecules can bend and move somewhat, and align themselves to a field. Comparatively, it would be easier to polarize.
This is just a few things that are at play. The question itself is quite open ended.
