I learned that a physical system with higher symmetry lowers the overall energy. If it is true, why we don't have many/all elements crytalized in simple cubic structure?
The energy of a crystalline solid structure depends on the type of interatomic interaction and a delicate trade-off between distances and symmetry of the crystalline structure. Moreover, one has to consider that at the freezing point, finite temperature effects (via entropic contribution) may stabilize some crystalline structures that are not the most stable at low temperatures. This is, for example, the case of the alkali metals freezing into a body-centered cubic (bcc) solid at normal pressure but undergoing a structural transition into a face-centered cubic (fcc) structure at low temperature.
Even limiting our attention to purely energetic considerations (therefore a zero-temperature analysis), the characteristics of the interatomic interaction plays an important role. For example, like in many covalent bonded materials, angular-dependent forces may favor open structures with less symmetry than more compact ones.
However, there are also elements whose crystalline structures at low temperatures have the highest symmetry. This is the case of many simple metals (i.e., s-p-bonded metals) having the fcc structure as their minimum energy structure. Notice that fcc is a cubic system Bravais lattice, as well as the bcc and simple cubic (sc). The reason for the dominance of the fcc lattice is somewhat related to the symmetry, although in a non-trivial way. Indeed, there is a higher symmetry of the fcc lattice, reflected in the highest number of first neighbors (12) of this structure compared with the others (8 for bcc, and 6 for sc). An intuitive way to see the higher symmetry of the bcc and fcc structures is by using a conventional description in terms of a cube with a basis for all the cubic lattices. In addition to the symmetry of the cubic network, the bcc structure adds the translation by half of the diagonal of the cube, while the fcc adds the translations by half of the diagonal of the faces. Actually, if I recall correctly, among the elements, only Polonium has the sc as a stable crystalline structure among the elements.