For some reasons, it seems that particular "handedness" are chosen by nature, because we have the prevalence of some enantiomer, that have different optical properties respect to polarization. In nature L-amino acids are prevalent respect to D-amino acids and D-glucose are prevalent respect to L-glucose. Is this breaking of symmetry a poor accident, in both distinct cases, or could there be a deep physical reason that caused it to evolve?
How do we know?
We cannot know for sure, since this stage of the evolution happened long time ago, leaving no fossil evidence. The main sources by which we can reconstruct the early evolution are the idiosyncratic features, like the molecular asymmetry, the sequence of steps in metabolic cycles, the protein configurations and random features of certain parts of DNA sequences that are common to all living organisms (which are not many).
It is likely has happened by an accident, although not necessarily a poor one. In fact, one should consider in this context two separate questions: the emergence of the first molecules and their subsequent evolution.
From chemistry we know that, when synthesized from achiral compounds, the molecules of alternative chiralities appear in approximately equal proportions (which poses a serious issue when producing medical drugs). Thus, it is quite possible that at the time the first self-replicating molecules emerged, there were ones of both polarities. We can say with high confidence that the chemistry (and thus also physics) prevented them from mixing, so the two forms of life likely existed in parallel.
Evolutionary theory tells us that a genetic diversity is removed from the population for hard probabilistic reasons, with one allele (in this case one polarization) becoming fixed. (See fixation.) Note that this does not require that one allele has an advantage over the other, in what is known as neutral evolution. Indeed, the need for a selective advantage was the early Darwinian belief, but the evolutionary theory has significantly evolved since then - notably in mathematical respect (see, e.g., Gellespie's little book). Now we have good reasons to believe that a significant part of evolution (although definitely not all) is a result of completely random events.
Moreover, modern evolutionary theory claims that all the currently living organisms have descended from a single organism, LUCA (lust universal common anscestor, see Common descent), likely inheriting its chirality.
Other idiosyncratic features
As another example of randomness easily understandable to physicists, one could mention the protein folding configurations. All the known proteins fold in about a thousand of possible structures... out of billions available in each protein's phase space. Moreover, these configurations are often identical between the proteins with completely different sequences of amino-acids.
Life as spontaneous symmetry breaking
A more physical (although less biologically concrete) way of looking at this problem is to view life as a result of spontaneous symmetry breaking in non-equilibrium systems (see the article by Anderson and Stein). In this sense the chirality of organic molecules is but one of the many components of the complex order parameter characterizing the "life phase". One could thus say that this breaking of symmetry is indeed an accident... which is a manifestation of the deep physical reason.