Is ensemble a set of all microstates of a system? Definition of ensemble says it's a collection of identical mental copies of a system in which microscopic parameters can differ.
Now, the phase space of the $N$ particles in the system in 3 dimensional physical contains all possible positions and momenta, $\{q,p\}$ of those particles in that system. So, can we think of each point in phase space as a microstate of the system?
Then geometrically ensemble is the phase space, right? I am very confused about how ensembles and microstates are connected.
 A: Phase space and ensemble are different concepts.
It is true that a point in phase space corresponds to a microstate but an ensemble is something different from the set of all the microstates.
An ensemble is a conceptual tool required to provide a base for assigning probabilities to the microstates. In a frequentistic approach, the probability coincides with the frequency of an event. Ensembles are there just to provide a way to speak about the frequency of every single microstate. It is clear that in the phase space points are unique: a single microstate appears only once. In order to have more occurrences of the same state, one introduces this concept of $N$ mental copies of the system. If a microstate appears $n$ times, it will have a probability $n/N$.
A: Like you said, each point in phase space is a microstate of the system. I like to think of an ensemble as being a probability distribution over phase space. This distribution by construction will be a maximum entropy distribution. What separates different ensembles is the set of states over which the distribution is defined and the entropy is maximized. For example, in the microcanonical ensemble we we maximize the entropy fixing particle number and energy, in the canonical ensemble we fix particle number but not energy (since energy is exchanged with the environment), and in the grand canonical ensemble we fix neither energy nor particle number.
