Gauss's Law involves doing a surface integral of the electric field over some surface. There are generally two ways in which you can use it:
The usual direction: given some distribution of charge, find the electric field; or
The "backwards" direction: given the electric field over some surface, find the enclosed charge.
The "backwards" direction is straightforward to apply: just integrate your known field and be done. But this is almost never used, because you're almost never given the electric field outright without knowing anything about the charge. Much more commonly, you're given a charge distribution and asked to find the field.
The main issue with this is that Gauss's Law only gives you the surface integral of the field. There isn't really a well-defined general inverse operation for surface integration (and if there was a general inverse, it wouldn't be unique by any stretch, because many, many vector fields have the same surface integral over some surface). As such, the best you can usually hope for is that the surface integral reduces to a case where you can deduce what it should be without having to resort to sorting through an infinite number of equivalent vector fields to find which one is a valid representation of your particular charge distribution.
This is where symmetry comes in. If you know something about the symmetry of your problem, you can constrain the formerly-infinite set of vector fields with identical surface integrals to just one, i.e. the only field that would preserve the symmetry of your setup. If you also happened to choose your surface so that the field was constrained by symmetry to always be perpendicular or parallel to it, then your work is basically done - the surface integral will be trivially analytically doable, which allows you to get the right electric field.
In short, due to the fact that surface integration is not one-to-one, Gauss's Law will usually only tell you something useful if there's some symmetry of the system that you can exploit.