Could the Periodic Table have been done using group theory? These three questions are phrased as alternative-history questions, but my real intent is to understand better how well different modeling approaches fit the phenomena they are used to describe; see 1 below for more discussion of that point.
Short "informed opinion" answers are fine (better, actually).


*

*If Dmitri Mendeleev had had access to and a full understanding of modern group theory, could have plausibly structured the periodic table of chemistry in terms of group theory, as opposed to the simpler data-driven tabular format that he actually used?

*If Mendeleev really had created a group-theory-based Periodic Table, would it have provided any specific insights, e.g. perhaps early insights into quantum theory?

*The inverse question: If Murray Gell-Mann and others had not used group theory concepts such as $SU(3)$ to organize particles into families, and had instead relied on simple grouping and graphical organization methods more akin to those of Mendeleev, is there any significant chance they could have succeeded? Or less speculatively, is it possible to create useful, concise, and accurate organizational structures (presumably quark based) that fully explain the particle data of the 1970s without making any reference to algebraic structures?

1 Background: My perspective on the above questions is to understand the interplay between expressive power and noise in real theory structures. One way to explain that is to note that mathematical modeling of data sets has certain strong (and deep) similarities to the concept of data compression.
On the positive side, a good theory and a good compression both manage to express all of the known data by using only a much smaller number of formula (characters). On the negative side, even very good compressions can go a astray by adding "artifacts," that is, details that are not in the original data, and which therefore constitute a form of noise. Similarly, theories can also add "artifacts" or details not in the original data set.
The table-style periodic table and $SU(3)$ represent two extremes of representation style. The table format of the periodic table would seem to have low expressive power and low precision, whereas $SU(3)$ has high representational power and precision. The asymmetric and ultimately misleading emphasis on strangeness in the original Eight-Fold Way is an explicit example of an artifact introduced by that higher power. We now know that strangeness is just a fragment -- the first "down quark" parallel -- of the three-generations issue, and that strangeness showed up first only because it was more easily accessible by the particle accelerators of that time.

2012-06-30 - Update on final answers
I have selected Luboš Motl's answer as the most persuasive with respect to the questions I asked. If you look at the link he includes, you will see that he has looked into this issue in minute detail with regards to what kind of representation works best, and why. Since that issue of "what is the most apt form of representation" was at the heart of my question, his answer is impressive.
With that said, I would also recommend that anyone interested in how and to what degree group theory can be applied to interesting and unexpected complexity problems, even if only approximately, should also look closely at David Bar Moshe's fascinating answer about an entire conference that looked at whether group theory could be meaningfully applied to the chemical elements. This excellent answer points out a rich and unexpected set of historical explorations of the question. If I could, I would also flag this as an answer from a different perspective.
Finally, Arnold Neumaier's answer shows how a carefully defined subset of the problem can be tractable to group theoretic methods in away that is predictive -- which to me is the single most fundamental criterion for when a model crosses over from being "just data" into becoming true theory. And again, I would flag this one as an answer if I could.
Impressive insights all, and my thanks to all three of you for providing such interesting, unexpected, and deeply insightful answers!
 A: The work by Kibler mentioned by David Bar Moshe is interesting but basically constitutes intelligent guesswork without a substantial theoretical basis.
On the other hand, there is very interesting group theoretic work on the periodic system by Gero Friesecke, based on first principles. See arXiv:0807.0628 and arXiv:0905.1236. His group actually computes orbitals, though in a not quite realistic limit, and his methods are mathematically impeccable. They only get part of the structure of the periodic system, but that in a strongly justified way. 
Edit: I just discovered a 2009 paper by Friesecke and Goddard in which they even use an extension of their method to predict level misassignments in NIST database tables!
A: No, the elements of the periodic table don't form any representation of a group or, more precisely, any irreducible representation. Even more precisely, the real insights by Mendeleev – that the reactivity etc. is a repeating function of the atomic number – doesn't follow from any property of a representation that could be derived by group theory.
The periodic table boils down to the electron's filling the shells in the atom, quantum states that are close to the energy eigenstates of a rescaled hydrogen atom. The closest thing to your project that actually can be done is to solve the full hydrogen by the $SO(4)$ symmetry – the rotational symmetry enhanced by the Runge-Lenz vector:

http://motls.blogspot.com/2011/11/hydrogen-atom-and-so4-symmetry.html

This solution dictates not only degeneracies but even the energies because the Hamiltonian is a function of a Casimir. And these energies are important to determine which $Z$ produce more reactive elements. More complicated atoms don't have any $SO(4)$ symmetry, only $SO(3)$, and they can't be solved purely by symmetries. The eightfold symmetry is useful because the elementary building blocks are numerous and they carry various labels – like quarks come in different flavors. But that ain't the case of atoms in the approximation of chemistry or atomic physics for which the nucleus only matters when it comes to its charge, i.e. $Z$, and electrons are the only other particles that matter, without any flavor indices. So there's simply no room for eightfold symmetries etc. The fundamental symmetry between elementary particles is $U(1)$, not $SU(3)_f$ as it is for light quarks.
If we neglect the electron-electron interactions in the atoms, we get another solvable problem – one in which we literally fill shells of the Hydrogen atom. This system is a second-quantized Hydrogen atom of some sort and it is solvable. We could say it is solvable by group theory. Of course, this approximation ultimately leads to a wrong ordering of shells and the predicted periodic table would be wrong for high $Z$, too.
To conclude, physical systems that may be fully solved just by group theory – and even properties of physical systems that may be determined by group theory – are rare enough, a small enough minority of the questions we may ask. Atoms are complicated enough so that their properties mostly boil down to more complicated dynamics than just symmetries.
A: Peter Thyssen from Free University in Belgium wrote an interesting paper about group Theory in Periodic table properties.
A: Although there is no known group representation which encapsulates all the properties of the periodic table, there are,  however,  attempts to gain a representation theoretical understanding of the periodic table at least qualitatively and there are recent works
mainly by M. Kibler in this direction, please see the following two articles arXiv:quant-ph/0310155, and arxiv: quant-ph/0503039.
The basic idea is that certain properties (such as spectra) of complex quantum systems can be approximately understood using models based on dynamical groups. A dynamical group (please see for example the following article collection on the subject) is defined to be a group which the system's phase space is a coadjoint orbit of. Using this method, the spectrum of the hydrogen atom for both bound and scattering states can be 
computed exactly from discrete series and principal series representations of $SO(4,1)$ and $SO(4,2)$, both of them are dynamical groups of the hydrogen atom. Also, the group $SU(1,1)$ can be used as a dynamical group generating the harmonic oscillator spectrum.
In more complex situations a clever choice of a dynamical group, descibing the important degrees of freedom of a system in a given situation, can give qualitative and even semi quantitative predictions of the complex system dynamics, an example is the use of the group SU(6) to describe collective nuclear states, see for example: Arima and Iachelo. There are also applications in condensed mater physics and particle physics.
Now, the dynamical group selected by Kibler is the group $SO(4,2) \otimes SU(2)$. The group $SO(4,2)$ was chosen because it is a dynamical group of the hydrogen atom and the group $SU(2)$ describes the spin. The direct product structure indicates that this model does not take into account the electron correlations. 
A specific infinite dimensional representation of $SO(4,2)$ gives orbital fillings very close to the Madelung rule. Kibler believes that it is possible to extend this work and arrive to a model explaining more chemical properties of the elements, such as ionization energies, electron affinities, specific heats and other properties.
In conclusion I think that the use of representation theory, especially of noncompact Lie groups can provide valuable approximate predictions for this problem as well as for other complex problems in quantum mechanics.
