How were non-Euclidean manifolds applied to physics before Einstein? In the letter of introduction to Einstein's 1916 paper on General Relativity, he writes, 

The mathematical tools that are necessary for general relativity were readily available in the 'absolute different calculus,' which is based upon the research on non-Euclidean manifolds by Gauss, Riemann, and Christoffel, and which has been systematized by Ricci and Levi-Civita and has already been applied to problems of theoretical physics.

In what way had it already been applied to problems of theoretical physics?
See this for the quote.
 A: The Stanford Encyclopedia entry on "Nineteenth Century Geometry" is not quite what you're asking for, but it seems nonetheless likely to be of interest. It's always worth trying this resource for History as well as for Philosophy.
I also found this in Google books, http://books.google.com/books?id=r9C-SCXymPoC&lpg=PA93&ots=LhLRqi5Fkh&dq=geometry%20before%20einstein&pg=PA109#v=onepage&q=geometry%20before%20einstein&f=true, which is considerably closer to an Answer, but still not quite what your Question asks for (you can see in this link that my search term into Google was pretty simple-minded, "geometry before Einstein"). This reference makes it seem as if there was a lot of speculation in the air, perhaps enough that Einstein and others, particularly including Hilbert, would have taken it for granted that geometry was where the story would be, but there's still no sign of the smoking gun.
EDIT: It is perhaps noteworthy that in my pursuit of a more definitive Answer to this Question I find myself stumbling into the substantial literature on one of the great controversies of the last 15 years in the History of Physics, the question of priority for GR, between Hilbert and Einstein. Reading a review of this sometimes acrimonious debate in http://arxiv.org/abs/physics/0504179 (it's a colloquium talk), one is further struck by the extent to which geometry is absolutely in the air for these guys.
EDIT(2): What I think is a better citation (by searching the PhilSci archive): http://philsci-archive.pitt.edu/4377/. The discussion here includes the following, on page 42: "Early in 1914, in a joint paper with Lorentz's former student Adriaan D. Fokker, Einstein reformulated Nordström's theory using Riemannian geometry (Einstein and Fokker 1914)." That it wasn't yet successful Physics doesn't matter, it has already been applied to problems of Physics. Good enough?
A: It is difficult to know exactly what Einstein had in mind, but the following is the first application of non-Euclidean geometry to our physical world. From wikipedia: Carl Friedrich Gauss, a German mathematician, was the first to consider an empirical investigation of the geometrical structure of space. He thought of making a test of the sum of the angles of an enormous stellar triangle and there are reports he actually carried out a test, on a small scale, by triangulating mountain tops in Germany.
A: The work of Ricci and Levi-Civita, what is also called absolute differential calculus, was less than ten years old when Einstein started working on general relativity. There isn't much room for other applications. I suppose what he meant was that it was accessible to everyone, not that it had been around for ages. 
This reminds me of another quote by Hilbert. 

Every schoolboy in Gottingen knows more about higher dimensional geometry than Einstein, but chalk is cheaper than gray matter."

