Is Pauli exclusion or not reason for formation of matter in the Universe?
It is not the most important, but yes, it is one.
There are some interesting phenomena that allow the stability of material structures.
First of all, there are the electric charges electrons and protons with their different masses and magnetic moments. While the electric charges electron and a positron (proton/antiproton) annihilate each other, this happens for electron/proton only up to a certain distance and then the emission of EM charges comes to a halt.
Secondly, this is the stability of the photon. Photons "inherit" both the electric and the magnetic field of the subatomic emitters during their emission and wobble through space with these two field components until they are absorbed again.
The Pauli principle as a stability factor of our surrounding world is based on the observation that in atomic structures electrons, when they occur in pairs in shells, always take on opposite spins. It is by no means absurd to associate this with the magnetic dipole of the electron and to imagine these oppositely oriented dipoles for a better understanding of the processes.
Now, QM models fantastic shapes of electron residence probability that suggests electron volume interpenetration. On the other hand, chemistry visualises precisely positioned electron arrangements to represent molecular compounds. This is theory versus practice.
Short answer: While the prevention of the infinite approach of proton and electron is the basis for the stability of material structures - under our gravitational conditions!, in neutron stars it looks quite different -, the presence of magnetic dipoles or the spin in the arrangement of electrons in the atom leads to additional margins in the stability of the structures surrounding us.
Is it referring to common physical space for all those fermions, and doesn't matter their locations are same or different, but they simply cannot exist in that space as long as their quantum states are identical?
Outside the atomic bond, the Pauli principle does not play a principal role for electrons; here, electrical repulsion prevails, preventing spin interaction. However, electrons interacting across potential sinks can be used for spin electronics purposes.
Free electrons can take any direction of their spin and thus completely coincide with other electrons in their state (but always locally different).